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ATJSJRACT | • * 

I This is the f irst JLn a k /5^^e6 of ^poxt^ vhiclt * 
snaaarize research studies, dealing tdth College level econoaics 
\ education that . is personalized, individualized, or self paced*. The 
,aajor purpose is to infoxa interested economic. educators of the 
^studies that haVe heen done, to suasarize .the results, and to. 
v jL3eati£y various research designs and statistical techniques that 
others aight want to use* Forty studies are described* Host wer£ done 
in the 1970s* Th^ introductory 15* pages give an overview of the 
research. Host, studies indicate that personalized or individualized 
instruction is as effective as the traditional iectura discussion , 
apprdash. Sany suggest that prograaaed materials s^vVtiae for Both 
s-tnden^ and instructor and can be _less expensive than the traditional 
a^proatfhes. ~T&e use of' television, coap.it er assisted instruction, 
tutors, and ^Ase studies is tested in several studies, and^soae deal 
#£th various coabinations of approaches. Soae; researchers 'believe 
rfiat- no single aethod is best, , but th^t certain^ coabinations sufih as % 
prograaaed instruction with gases and siaulatibns or with an audio 
visual tutorial systea yield superior results. {Author/RHJ 



A 



v 



Document* Stqutr*d by EK'ICincL if <«uft/ ir.f„nr.al 4npabi»h»d mjfttdu n »t wutabl* itvm clL*t vourc<» EKIC mafct. > ♦fvvr/ 
♦•ffort to obtain the host copy ivailabte i:*.ertfc*l's* am,* of iuit^nal rtpt^iuotmity at* often. *ncouMi»rc4 and um MU~\$ ihn 
quality of th* -titarcflchi* and hard ^ / rcpod" tiotw LPIC mar.** dvatlatlr tf* LhlC Document Reproduction Service »LtmS). 
Erys |5 not res^onsibto for thr quaiit/ A thr* c#rtgirui i^m^ui > *\>tA vM.rrx wppitf-i b-» ED^.w thr b^t tn^i can be mad< Uzm 



CO 

W * 




, Mi Dff»A«TA*e*TO*MtAtTH, 
SOVCATiO* 

*£*t o^ncut w.O'ut u^ftitute cr 
t Ogc a? ion r»ox o^5v* cv * 



A SUMMARY OF RESE&RGH IN 
PERSC&ALIZED, INDIVIDUALISSEb , AND SELF-PACED 
INSTRUCTION IN COLLEGE ECONOMICS 



George G+ Dawson 
i 



P«RMl^3lO^ TO H£PHODUC£ THIS 

TO THE fGUCATlOftAU RESOURCES 
WOhttATICP* CCVTFR CtRIO AND 
THH £«»C SYSTEM ^ON-TflACJORS 



Research Report* No. 1 _ . _1 

of 

The Center for Business and Economic Education 



Empire State College 
Long Island Regional Learning Center 
Old Westbury, New York 
^ 1976 - 1977 



Published with the assistance of the 

New York State Council en Economic Education 
State University College * 
Geneseo, N. Y. , 



ERLC 



A SUMMARY 0% RESEARCH IN PERSONALIZED , 
INblVIDUAI.iZED/ AND SELF-PACED 
INSTRUCTION IN COLUEGE ECONOMICS" 

by. ^ • * 

George Dawson • 



INTRODUCTION 



This is the first in a* series of reports on research in * 
'' • ' » ' " \ 

economic education being prepared by the Empire State College 

■Center for Business and Econonlic Education. The CBEE was'es- 

tablished late in 1976. Upon becoming officially affiliated • 

vwtth the New York State Council «on Economic .Education and the 

Joint Council on Economic Education in February'of 1976, the 

""**'•.». *'". .* " ; " * 
•CBEE was designated by the Joint Council as the "National 



CenEef for Personalizing Instruction in Business and Economics." 



(See Page 4 



5f Progress xn Economic Education , Volume 7, 



Number 5,. September, 1376, published by the Joint Council.) 
Shortly thereafter, the CBEE produced its first publication,* 
Personalizing Instruction in Business, Economics and. Related 
Stfttfects , Complimentary 'copies o£ tliis 56-page report were sent 
tp'Svery affiliated Center and Council in the United States." 

^Interest in personalized 1 , individualised/ and self -paced 
instruction seems to be growing. Many college teachers are- 
raising questions about the efficiency of the'qld "lock-step" 
curriculum, an educational situation in which students in a 
given course are exf <3cte< to move at the same pace, using the „ 
sam§T (or similar) materials, and bein$ exposed to the same teach 
ing techniques*. We do not expect all students tb wear the same 



size shoes,, but, somehow we often think that a prescribed cur- 

riculum and method. of instruction should "fit 11 every individual. 

Several factory probably helped^ to motivate college economists 

to try hew approaches , to experime nt wii-hf unconventional ec|uca- 

tional materials and equipment /Itnd to recognize that what is 

right for one student may be very wrong £o;r another * Reports 

jfchat students were avoiding economics f that there was little ^ 

evidence of the residual impact df-j&c^nexrics instruction , and 

thit the student uprisings of a few years ago were caused (at 

/ least in part)., by dissatisfaction with pur teaching may have 

contributed. Furthermore/ economists habitually consider 4 the 

c/Ssts 'and benefits of everything 'else, so tfhy not attempt to 
.v > ' . 

Measure the costs and benefits of economics bourses? In. any 

event/ the 1970 r s have seen a dramatic increase in research in 

the teaching of economics at the colle^e^level. At least *750 

studies of one kind w or another, have' been made of economic 4?du~ 

cation for college students K or adults / and ov*r 500 of thos& 

appeared in the 1970* s. (See George G. Dawson , "Spepxal^Report: 

An Overview of Research in the Teaching of College Economics , n 

The Journal of Economic Education / Spring/ 1976/ pp. 111-116*) * 

As part of its obligation in serving as tne national center 

for information on personalized instruction/ the Empire §tate 

College CBEE began to collect studies in any way involving S<3I3f- 

paced/ individualized , or personalized ijjgihods/ Lists of re~ 

search were examined,, and about 60 titles were identified thut 



seemed in some way to suggest relevant studies* Upon examina- 
tion/ some of these papers proved to contain little or nothing 

on the subject. Others, merely represented duplication* That 

J 

is, .soiue researchers produce two or more papers on the same 
project, perHaps changing the title aid modifying the text, but 
reporting nothing new or. different. Efforts "were made to ob~ A 
tain copies of every study that was uncovered, but the success 
rate was less than* 100 percent. It cannot be, claimed, then, t 
that this report contains ev^cything. that .has been done on our 
subject. Readers a*e urged to apprisp us of anything we have 
missed or overlooked. *' 

The fact that, a study is included ''in this report does riot 

0 * ... ^ 

imply that th§ Center considers JLt to be an exqellenj: piese of 

research. „The research vairies In qualify and importance. Our 

« . » - * 

major purpose is to inform interested economic educators of the 
studies that havfe been done, to summarize the results, and to 
identify various research designs and statistical techniques 
that others^might want to use. It is hoped £hat this bodklet 
will help other researchers who wxslTT^4urvey" th^ . literature, it 
the field, saving them considerable time and effbrt. Remember, 
however, that no study has been presented in its entirety. In- 
deed, we have reduced one 200-page thesis to. a paoe and a half I 
Our summaries may help the reader decide which papers to obtain 
for detailed analysis, however % Those planning rssearch in per 
sonaiized, individualized, or self -paced instruction should 
examine the work of others, noting their succ6s#es and failures 



and their strengths and weaknesses. J _ 

^ What ,do we know from research that has been completed, 
andS/hat elVe needs to be done? £irst, there%s ample evi- 
dence that self -paced, individualized , and personalized in- 
sti^ctixu^ cjui J3& JiJ^ _ _.. 

students. Pe£r (pp. .36-3,7 below) and Spector (pp. 45-48 below) ' 
found the Keller method of personalized instruction to be 

- { * • . 

. ' ■ r 

superior to conventional methods in undergraduate classes, 
while Roop*s study (pp. 37-38 beiow) of contingency management 
indicated superiority for that apr^oach, over conventional tech- 
nigues in an intermediate microeconomics course. On the other, 
hand, Soper and Thornton (pp. ,43-44 below) found self-pacing to 
be inferior. Host others find the various forms of personalized, 
individualized, or self-paced instruction ,to be* as effective as 
the traditional lecture-discussion approach , 'at least "as measured 
by objective tests of economic knowledge and understanding. 

Many suggest that programmed materials and some of the, other 
techniques save time for. both student and instructor. That is, , 
students can often learn some of the basic economic principles on 
their own, s thus enabling the teacher to use class time to appi^ 
the concepts to economic issues and problems, and/or enabling the 
student to devote more time to other studies. For examples of 
the time-saving argument sec the Attiyeh, Bach, and kurasden study 
•(pp. 3-4 below) and the Wetzel study (pp. 55-56 below) * v 

The value of lectures -is questioned in several studies. The 



Suckies-'-icMahon paper (pp. 12-13 below) deals with this issue f 
and; McConnell and Lamphear {pp. 28-29 'b*elow) assert that lec- 

. "-• '"■ '-. ' .' ^ ■ : . - . 

•tures are not always necessity even for the .poorer, student. -It 
is dangerous to generalize* however, # because the studies re u 
* ported here cover a wide v&ri.ety of situations, practices, and" 

'materials. . • f. ' • 6 

i ' ' • / ' 4 :■ " • 

The use of programmed materials is dealt with in many' of 

•' . . / ■' >• •'■ \ 

-the^sjtuaies,. and the* usual finding is that programmed textbooks f 

are effective in teaching economics. Havrilesky w^irns, however/ 
that we must consider, the time dimension. (See p. 22. below*) 
'That is, programmed materials, may be effective when used for 
'short periods ; but fchefr impact might decline if the period of 
use is extended. Thij is a question that definitely, calls for 
additional resea. h, . The popularity of programmed materials 
with, students is still a matter of dispute\-~Uurasden reporting 
them ,to be most- popular (see p, 28 below) > while^ Luker finds 
them to be least popular (see p. 27 below). .. .. 

• The use of television,. computer-assisted- instruction, ^tut'drs, 

and cas'e studies is also tested in, several studies, and some deal 
with Various Jiorabinatiohs of approaches/.. In fact, some re- 
searchers believe that ho single method is^ best, but chat certain 

t * + * ft 

combinations . yield superior results. linker, for example, tried 
four teaching methods and concluded that a combination of pro- 
gsfammed instruction with games and simulations worked best. (£ee 
pp. 26-27 below.) Tolles and Ginman.also thought that a combina- 
tion — in this case an audio-visual- tutorial system' with 



cd^yentiooal class, discussion methods Was superior. (See 
p ? " si Below.) This, again/ is another area in which much more- 
\ .research is required, for the number of possible combinations " 
Ifera^&St limitless . . j 

experimental techniques as comp&rejd with the costs of conven- 
tional instruction* This is particularly important where one 



finds no significant difference in effectiveness between methods. 

^ Generally, it has been found that, several of the p^sonalized, 

_jL:L; -a :; - . ? , > . j ».* - 

individualized # or self -paced schemes* can be less expensive than 

traditional approaches* Hellish and Bostow (See pp. 29-30 below) 
^ * * . „ , - - Y , 

claim that ttfeir use of^tftg Keller method pf personalized in- 
struction cut v U»e per-student cost by about fifty percent. % Booms 

and Kaltreider, on the other hand, found that their use of com- 

• » 

puters increased some qf the unit costs. (See pp. 9-10 below.) 
Some consider c6sts in terms of student time as well as money 
r and other college resources expended. Becker and Salerfijl (see 
pp* £-8 below) found that there^were savings in terms of. the op- 
portunity cost of student ttme. Allison (pp. 1-3 below), treats 
seEf-paced instruction as a new technology in. a production, pro- 
cess* Her study suggests that one must distinguish between the 
higher initial cobts experienced when a new program is being t 
. started arid t^ie costs oyer the longer run period. In any event , 
Researchers should include a cdst -output dimension in all studies 
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The lasting effects of the. experimental methods have been . 
<jot|siaejJed" by several researchers. The .period in question .is 
usually relatively short, however. The students are tested * , \ 
Igain a few months or a. year after the experimental treatment. .. 
iastjfchg effects were considered by'Booms and Kaltreider*{jpp. 
9->10 • below) /craig and O'Neill, (p. 15 below) , Kelley (p. 24 j 
below), Siegfried and Strand (p. 40 below) ", Spector in his se- 
cond paper (pp. 46-48 <below), and Tietehberg (pp. 4B-51 below)-. 
Craig, and O'Neill, thought that self-instruction aided in reteh- 

• tion, at 'least for^iho^. items An. the "cbmplex--application' t ^ 
portion of the of Understanding in College Scohbmicg. Stu- 

- dents exposed to Keller's Teacher. Information Processing System^ : 
(TXSS) maintained 1 their/ superiority after a year Jp. ^4 below),- " 
but Siegfried and Strand Cp. 24 below) found 'that students getting 
personalized instruction.. enjoyed no advantage 6yerjdonventlonl(X'ly- 

r % taught student^ even after; one semester, .Spector' s P.8»£* 
students did better when they got into 

*~ Tistenberg found no significant difference. . ?hus> "the jury is 

* still' out « p» the residual impact. of personalized instruction, 



especially for periods of overdone year. 

The disaggregation of da&a to test for differential effects 

P \ . 

on different types of students is a matter of considerable im- 
\ portance. Billings asserts/ that P.S.I* is' "rewarding" for all ' 
" ■kinds of students (pp. 8-#fbelow) , but Allison fuel's that self- 
pacing, is best for freshmen, and the "less brilliant. V-Those 



agreeing with Allison are Eusfeld. and Jump {pp. 18-19 belpw) , 
JKeiley (p. 2? .below) and O'Connor (p.- 34 below). On the other 
hand;, Havrilesky thought that his programmed text was more 
",. suitable for the better student <p. 22 b§jpwj.. Negus (pp. 30-31 
below.) also took this position, and Tietenberg tended to agree. 
Clearly, all future research should attempt to disaggregate the 
•data a Xt mUjSt not be assumed that a technique that appears to 
„be good for .all students as a group, or for students of a parti- 
cular level of ability, will likewise be good for students with 
differing characteristics. It should v be noted, too, that few 
studies went beyond .a. single college. The Attiyeh-Bach-Lumsden 
study (pp. 3-4 below) is pne of the few multi-school research 
projects, involving 4§ schools and over 4000 students^ -..Two-year 
j colleges have been largely ignored, except for the Becker-Salemi 



and Walstad studies* Where differences are found between colleges 
It is impo r tant* tc rtxy to ascertain the reasons. 

* Many studies have tried to measure student attitudes toward; 
the course and the method of instruction/, f inding;thaf students 
are often (but not always) favorably disposed toward the experi- 
mental techniques. Little or nothing has been don^ however, to 
ascertain possible effects of the treatments on student ^attitudes 
or opinions- toward "economic issues. This, indeed, is a wide-open 
arecpfor future research. ■* • , 

< 

The quality «and-characteristi<Ss of the teacher is another—,, 
matter-that ne^s^further stud£. ®*ch's study (pp. 4-5 below^ 
deals .with this, afjd Van 'Metre' suggests that the teacher's expec- 



JO 



tations and motivation might have a bearing 6n course outcomes. 

Indeed, Van Metre .warns that failure to consider the teacher. 

along with the methjod may be a great ^weakness ' in much of the 

research/ He is probably correct in noting that*a "given method 

.should not be considered in isolation — that a technique tl\at 

works well for dne teacher may not work well for another. Our; 
< * 

"experience at Empire State* College supports this. The^ESC mode 
of instruction calls for one-tq-one relationships between in- 

.. structors and students., with* the teacher as a guide and mentor 

. rather than an omniscient and dictatorial expert. Those who see 
t heir role a s telling students what to do, unilaterally estab- 
lishing degree programs and criteria for evaluation, and devel- 

.oping standardized lock-step curricula do not do well in this 

College* ' . * / 

. Most of the research has concentrated on the introductory , 
course la principles of economics/ However, O'Connor (pp. 33-34 

* * * t 

below) dealt with. students in an intermediate micro course (as 

did'Tietenberg) , while Roop's study involved those in intei^mediat 

.* /• ■ 

macro tpp- 37-3"8 below). A few included both introductory and 
intermediate levels, usually by considering the effects of the 
♦ experimental treatment at the lower level on the students after 
"they had reached the intermediate level. (See the dis assion of 
•"lasting effects" above.) * * 

Since some of the experimental techniques im#&ved the use 
of tjitors, the impact on those tutors has been a matter of con- 
cern^^IJppeT^iTiH~6^^ # are often used as tutors or 




proctors, " and some y£ind this practice to *be . effective and in- 
expensive. The student- to-student tutor iijg system described 

( /' * 

by Kelley. and Srfartz (pp.* 25-26 below) had a positive "impact on 

s/tudent performance. Seigfried .and Strand (pp. 39-41 below) , 

Using the 100-dtera* CLEP test in economics/ found that their 

futons learned much more through* proatorlng than -they would. 

have by taking an advanced economics course* 

Future r esearchers^ shni 7 Id pay^ particular attention to ithe 

research designs, statistical techniques, and testing procedures 

■ 

used in these studies.* Note the ptoblemg encountered by the 
authors, and their weaknesses as well as their strengths* First 
let us look at the< problem of testing* Which test does one use? 
.A standardized test such as the TtJCE ( Test of Understanding in 
College Economics ) has the advantage of having 'bfeen developed by 
h team of experts .and having norm data based upon nationwide ad- 
"ministration and validation of the instruments The major disad- 
vantage is that a standardized test may include material not 



covered in the ^oufse and may omit, some' things that were covered* 
Fnrl^ may measure knowledge, understanding, and perhaps 

ability to apply economic concepts to new situations, brut it 
does not reveal stucFent interest or attitudes toward the course 
or toward econ mic issues*. Anyone Jiot familiar with the TUCE 
should obtain a specimen set, which includes a manual , *^unples of 
the four 33-item forms^ and, sceritrg keytf. Vart X of tlTe^TUCE 



deals with bazlr Macroeconomics; Part II deals with microeconomics, 

There are two forms (A and B) for each part so that one can 

» ■ 
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administer. *a post-test that covers the same concepts found in 

& * ' ~ * 

. the pre-test without repeating the questions verbatim. This' 

test* was published in 1967 by the Psychological Corporation, 
... " * . * • v 

but .it is how old by the Joint Council on Economic Education, 

1212" Avenue of the Americas, New York, V.Y\ 10036. (The price 

' f . • • 

for the specimen set is $5.00..) There is also an unpublished 

" • * * • 

"Hybrid TUCE," a 33-item multiple-choice test drawn from the 
^ various' parts of the TUCE proper, and covering both micro and 

* macro • Those* wishing; to-use this 'shorter version of the TUCE 

Should * wri te *o the Joint Council's Director of College and s 

# - • • . *. *■ 

, University Program foi? a copy and for permission to use it*. 

Many of the projects reported in this booklet employed £he TUCE. 

Before the TU(?'E was qleve^oped/ many researchers used the 

* Test of Economic Understanding , published by Science Research 

../'Associates, the TEU was originally designed for use in senior 

high schools, but it has been widely employed with college stu- 

* * 

dents and -adults as well. This test has two forms covering the 

same bapic concepts. Th^re are 50 items in each form. It was 
published in 1964,. so the TEU is now considered to be out of 
"date, and the norm data are probably no longer appropriate even 
for secondary school students. i It is expected that the TEU will 

* be replaced in the near future by the Test of Edonomic Literacy 
bfeing developed under the direction of JoKn Soper at Northern 
Illinois University. S&per plans to draw upon a variety of 
sources (including, the college-level TUCE) in constructing^ this 
new instrument. 



» er|c. 



Some of the researchers developed tests . of their own to 

overcome the problem of using standardized instruments* that 

'might hot be appropriate for their particular classes- See,/, 

Seigfried and Strand .(pp«* 39-41 below)^for an example of this,,. * 

• - • 

The^problem here, of course, is that one cannot then compare * 

the results -for* a particular population with a natiqn-viide 

t 

normjLn'g sataple, or with results obtained by o^her researchers, 
who used a standardized test. Seme researchers have relied* 
at_lea-st partiaily, on a test-question bank maintained by the 
Joint Council* pn Economi Education. (See Buckles and j^cMahon, - / 
gp: 12-13. below.) ■ * * 

a The way in which a test is used may be as important as the 

> * 

test itself. The test should have an affect upon the student's 

h 

grade — otherwise students will not bother to exert much effort 
and the "test results will be meaningless. Hanni makes this-point. 
very clearly (pp. 20-20, be^pw) and even questions the usefulness - 
of the TUCE on the ground that some of the instructors administer- 
ing it to 'collect norm data may not have made it-part of the stu- 
4 dent 1 s final grade. 

Research design and statistical technique^ are stressed in 
s^vevral of the studies. Questions might be raised about those 
experiments in which students were permitted to choose between 
experimental and control sections, unless the researchers in some 
way adjusted for this. (See Kelley and SWartz, p. 25 bjalow, for - 
example.) ^Disaggregation is urged by several of the authors, 
and we hav$ already mentioned one way of doing this above. Note 
also that the TUCE items are divided into three categories — * 
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"recognition and understanding," "simp^ application," and 
""complex application. * It is suggested that researchers dis- 
aggregate the data in these terms, for one may find that an 
experimental method is more effective m promoting one type 
of learning as opposed to the two others. (See Craig and 
d'tfeill*' P« 3-5 below, for~ example. ) 

Readers concerned about research design might be particu- 
larly interested in Allisons' treatment of self-paced instruc- 
tion as a new technology in a prodUctitJoa proce?^ the multiple 
baseline and reversal techniques descried by Bostow and MeTlish, 
the problem of multicollinearity as discussed by Soper and 
Thornton (pp.- 43-44 below); and Tietenberg's ''breakeven point" 
GPA and use of discriminant analysis. Note the several dependent 

variables usedjb* O'Connor, and pay particular attention to £he 

• . & • ' 

*" gap-closing model." Man^i economic education researchers now 

use the gap-closing scd'ste as opposed to (or i?i addition to) a 

simple gain score, lue gap is the difference between the stu- 

'.dent's pre-test score and a perfect score say ten points, If 

the student's -gain score is five points, thetj the/ gap- closing 

score is 50 percent, Tl\e use of the gap-closing ntedel seems to 

have solved some of the problems encountered when one uses a 

simple gain score- (For a recent discussion of this ; , see Paul H« 

Kipps, Howard M. Wilhelm, and Daniel R. Hall, "A Mote on the Use 

• 

of Multiple Regression Analysis in Studies of Achievement in 
Economics," Th»? Journal of Economic Education , Spring, 1976, 
Volume. 7,- Number 2, pp. 130-132.) The Decker-Salemi paper is als 



1 



or interest , and one of the fascinating things about the ' 
Siegfried-Strand study is that, thb researchers addressed them- »• 
selves, to many of the criticisms made of earlier projects and 
to some of the questions raised fay previous research. The re- 
search-minded reader will also want to see the "Special Section" 
in the Fall 1976 issue of The Journal of Economic Education „ "A 

t 

Debate 'on Research Techniques an Economic Education-" 

If all of this research does not provide, overwhelming sup- 
port for those committed to personalized, individualised, and * • \ 
self-paced instruction, neither does it imply that they ought to 
be scrambling back to the lecture hall. Indeed, even those who . - 
found the experimental treatments to* be less effective " than^the ^ 
conventional methbds usually had something positive to say "about 
them* Ijt'is probably safe to conclude that these approaches con- 
stitute another useful instrument in £he college teacher's tool 
kit, and that their benefits have been fairly well established- 

Most of the research has been done £n thecontext of .a tra-' 
ditional institutional setting. Sometimes the experiment consumed 



I 



only a few weeks of a semester-long conventional course, and 
sometimes the treatment was administered in conjunction with the * *, 
traditional medicine. Fqw, if any, of the experiments are com- 
parable to the educational methodology employed at Empire State 
College- Nearly all students at Empire State College study ind'e*^ ** 
penaently, under the guidance of a faculty mentor. A dozen, students 
may be learning economics at the same t^ime^t one of our centers, 
but using different. textbooks and materials, doing very different • 

» * * 

lb , 



assignments (assignments often related in some way to their 
jobs), beiftg evaluated in different vays* x and proceeding at ^ 
rent rates of speed. Those of us at ESC think, this is a 



ver^ 'effective method of teaching, but we are still pressed by 
bur . colleagues elsewhere to "prove it*" .A model £qx program } 
ef fectiveness and related coots 1 (PERC) is being developed by 
•Ernest Palola, ESC 1 a Assistant Vice-President for Research and 
Evaluation* Pre liminary resear^^uggests that the ESC^mode is, 

in, general, both effectiye and economical* We do need further V 

• * - J ^ . * , 

evidence on the teaching of business and economics 9 hdwever, 
. Lloyd kill,o£ the ESC .Center^ in Rochester is making a study o£^ 
the cost-effectiveness of business ^programs at Empire State Col- * 
lege, anl Jeffrejf Sussman of the" Long Island Center Is planning 
a project to measure student learning of economics through, our v 
* system of personalized, self'-pacfed, and individualized instruct/ 
'tion* Meanwhile , we are fairly confideht that we are not wasting 
student tiraepr taxpayers 1 money. We are indebted to the many 
- .scholars y/hose work is reported in this booklet, And we hope in 
the future to be able to add substantially to the findings ^reported 
here/ • ' < , 
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^SU£1MARXES OF STUDIES IN j . 
^ , jCNSTRqCTIQN, IN* ECONOMICS? . * . 



$&lison, Elisabeth,.. "Self-Paced Instruction: A Review, n The 
ffotirrial of. Economic Education . Pall, 1975, Volumfe 7, NumEer 1. 
Pp. S-Vtr " 5 ^~ , • • * 



, S4 . Alison nofies .tjtoeit over, 80 cplleges and .universities have 
£dopt£#^alf-paced ^introductory . courses and predicts further 
growth in the future* ,She comments on thi* benefits and cost's, 
stiB^arlizing the results* of several studies* (T^hese studies will 
be described elsewhere in this report.) She concludes that *the . 
evidence on SPI's educational benefits is at present inconclusive*" 

, Finding that SPI has not demonstrated superiority ih raising stur 
dent, scores on standardized ecoHpmics tests,, she asserts tha,t. its 

^eis justified oh such criteri^as intuitive ^ppeai, student and— 

'teacher Satisfaction* .arid "ah intelfc&st in concentrators** 

^ — *' • ) ■ /■ ^ ' i • m / 

The costs of running, a selfrpace&^yst^m are identified, and 
^llison points out that these v$r% with tire format Selected. Costs 
, |>Qrne^hy the stud$nt_a,s well as^ ^y th^ instructor f and graders are 
£oMidered* fc The Steps f^r settihg up ah SPI* <iours£. ate l4£te& and 
9 discussed ^ including the defining of units, the writing- pf objec- 
' ti'ves,' the choosing of th,e t^st forrtiat, the preparation of other 
bourse nvateriais, ^e ^irin^ of graders, and the £hoosing\of a' 
grading, system* Allx5b*w<iitici2es> tho studies she reviewed for 
nqi: including cost figures 'and fcr not giving enough thought to , 
experimental design. ; 

<4 finally, ?he calls for disaggregating the 'data to test for 
differential effects on different types of students. To th$ ex- , 
tent "that SPI does increase learning, . there is a need to understand 



*********** 



Allison, Elisabeth, "T\ree Years of Self -Paced Teaching in Intro**- 
ductory Economics at Harvard," American Economic Review * May, -^1976, 
Volume 66, Number 2. Pp. 222-228. * 

Also see; Elisabeth Allison, "Self -Juiced Instruction in^ the Intro- 
ductory Economics Course," Harvard^ Institute of Economic Research, 
parier no. 368, 1974. 

.Allison identifies the benefits and costs of SPI, the way it 
changes the learning process, and which students find it most. 



helpful., She finds that "under the right, circumstances^ the 
&a*giri#l product tot SPI is a 10-25 percent increase in scores 

'and th # at it inspires stujdtents to take more courses in econdm/ . s. ,f 
{AER, p. 221.) On the other hand, the initial, cost of wStai^ 
lishibng the "right circumstances* is high, the benefits are^jot 
uniformly distributed, and after an initxcl period o£ enthusiasm, 
the students are no happier than they are in conventional courses*. 

f ■ • : ' _ * " ' ' - \ 

The Harvard approach was to prepare a list of operafcxonally 
stated coarse objectives aad a set of unit tests which the stu~' 
denies coulfiTtake whenever they believed they had mastered an 
objective. Upon failing an ex<ai*f the student pould take cpother 
test ori €he same material. There was no limit on the. number of 
attempts permitted, arid full credit was given When exam was 

"passed. SPI Was .introduced into three sections of about 30 stu- 
dents each, selected at randlom. . y \ . ■ 

* - * • * i ' " * ; 

The original intention was to follow $ conventifen&l evalua-> . 
tiph format — "estimating an achievement equatid^ in which a 
test score tfas the dependent variable .and SAT scores, age* Sex f , ... 
and gfa#e point average tGPA) , the, controls.. (XBR, p. 223.) 
This waiis\s crapped, however , in favoif of a schem# in which SPI 
would ]bre treated as a new techncflogy in a production process., 
^lligon <?oes not claim to have' found the best mpdel for evaluating 

/-SPI, for : there was a serious "errors- in- variables" problem, and 
they JTelt hampered fc by the lack of a jgenerall^ accepted theory of . 
.learning* ..... *' 4 ■ •' j 

'* \ . ** 

It was assumed that students allocate their efforts among 
courses and other activities xn such a way a^s to maximize some 
function, -Thus, a student choosing to devote some effort to Vi - % 
economics faces a production function defined by his or her ^bilr 
itJk,. teacher quality, and the instructional method (SFI) . Enjoy- 
ment of the, course will depend upon ^profit" th&\difference 
between the grade and the" cost (psychic ot financial)^ to, the/stu- 
dent, ^rhus, , equations were developed for effprt, for^hievekent, 

; and for enjoyment* Finally, the desire to take farther economics 
courses was seen $is an important output, so a concentration eqiia-t 
tion ("intended courses*') wasTalso added, "in short, the model was , 
designed to answer these questions: ( \ V 



"Does SPI have ^ny 
or concentration? 



net effect on learning, enjoyment, 



"How do these effects,., vary among students, i.e.*,* : 
does . . * 'SI 5 ! differ systematically among definable 
classes of students?"' s 

"If SPI has .any effect on achievement, enjoyment or' 
concentrations, what is the mechanism through which 
it works?" CAER,. p. 225.) / • 

*» 

The somewhat disappointing result was that "The direct contribu- 
tion of SPI to student happiness is insignificant and its con- 
tribution to concentration decisions is small." (AER, p. 225.) 

. * : ' 



Although J&.e . fi^st year o£ the experiment produced "both r<^- 
^IrJcably students and a very high "proportion of cojlc^n- 

tratpfcS/ 11 the second and third years did not -~ suggesting that 
the Hawthorne effect and y heavy faculty involvement 0 during that 
iirst yearJbad been instrumental; , 

' * ■ 4'** ' * v 

On the other kan<|| SPI did appear to improve performance . 
(by about 15 percent) on a multiple choice final examination 
|mad$ up of 10 TOCE iteras, 20 GRE questions, and 10 lodral items) . 
SPI substituted fpr £bo.ut 150 SAT points and 10 IfiOurs of .weekly" 
preparation by arT instructor . ,ln regard to the question of vho 
derived the greatest benefit from Si?&, Allison found that f in ^ • , 
general, lower SAT groups and freshmen eri joyed a slight advantage 
ov^r higher JB&T, groijps an# upperclags student? f She concluded: 
"the marginal product of the well-prepared teacher, is highest with 
the 1$bs brilliant Student;*- (AE&; ~p. 226* )>" ^ * % 

, ; J{f#y 444 s° me students do better " in SPI? It. was found thafy 
V*.the crucial feature H of SPI is the interaction between students 
and % grad6r; t that It /is not just glorified homework but & very di-7 
xectied^^prm of one-to-one instruction." Allison concludes "We 
se.exflj, to have allowed pur students to substitute efficient,, inter- 
active hours for passive or unfocused hours although apparently not 
to their great pleasure*" She further asserts that "..•moving to a 
SPI system brought about enormous improvements in our introductory 
course quite apart from, its contribution to the learning, of ^parti- 
cular stuclents." fAER, p; 227.) *' ' - ^ ' 

' Wison's work is deserving of considerable aU^tiba? ' 
because of the findings of this study and the^ interesting research 
design she .employed. ' % ~ . ; . C 



\ 
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Attiyeh, Ricihar# E. fj G*L. Bach, and Keith Lumsdeft, "The Efficient 
of Programmed learning- in Teaching Economics: The Results cf a 
Nationwide Experiment," Am erican Economic: Review , May/; 1969, 
pp* 217-223. \ 7- ; ' r 

j ' ' \ * ^ 

^ Over 400*0 students' in ^40 schools wer£ involved in this ex- 
periment in the use, of programed materials in the introductory * 
college economics cpurso. One group of students ut -d ]tas>grammed 
.tex^>ooks exclusively. A second used the programmed materials, 
but \ also, attended conventional lecture classes and discussion ses- 
sions. The third $roup received conventional instruction and did 
not , use the pfofrajpmed books. Tv;o programmed books were ufeed, 
yielding, different, result^. The.grdup using these I^ooks exclu— 
siv<f£y did not do jas well as the, other two groups. In the grQUp 
using one of these books afe a supplement Co conventional instruc- 
tion, a statistically significant advantage was found. The other 



t\4 t 



book did not mkke a* Signi-jficafit difference* 1 

1 

A aajor finding *was 9 that students spending kn average of 12 
hours studyipg a progrsoraned text could learn about as, much as 
t^iose- receiving jsevfefi weeks of conventional instruction. 6 Further- 
more, students using .only, the programmed materials performed better 
on questions mailing for the application of theorly than on simple 
"concept recognition* items, and had jl positive attitude toward 
programmed learning. * ~ " 

Similar results were reported in "factor's. Affecting Student 
Learning of 'Elementary Economics^" Keith Lumsden, Ed. , Recent 
..ReaeardLJLaJ^ (EnglewoSd ^iffs, ft. J.: Prentice- 



Hail, 1970). ,Also seer Keith Lumsden,. "Technological Change, 

Sufficiency, and Programming* in Egonomic^ Education, 11 in Keith ^ ^ 

Lumsden, Ed*,, New Developments in the Teaching of Economics "(En^Ie- 
wood Cliffs, N.J. : Prentice-Hall, 1967, pp. 27~~58*, and his "The 
Ef £^ctiveness=e£ .P^^r^mrhed Learning in Elementary Economics,'^ 
American Economiq, Review , May, 196?, pp.652-659. * m . \ 

* questions about the Attiyeh,, Bach, Lum^d^n study were raised 
by one of those"^ authors, however. G.L. Bach, -in "''A Further Note 
-on Programmed^ Learning in Economics," The Journal of Economic 
Education , EJerl^, 1969, Vqltune 1, Number 1, pp. 561-59, noted that 
the stu$y had hot included* an adequate v^tiable for tHe~ quality 
tho-teacher^ To explpre. this^ -an economics instructor at / 



Stanford University who was noted for t>exng one of the best y 
teachers in the economics department was Used in a controlled ex- ♦ 
periment. Students were given the choice of (1) using only the 
programmed textbook and not attending classes; £2) reading only 
the regular textbook and. not attending classes ; j or (3) using the 
regular text and' attending classes. The students learned a little 
more in the coaventional mode than b^ using the programmed bctok 
only* Those studying the regular textbook did as r well as those 
using the programmed material, suggesting the possibility that the 
students in the national e*pe*riment using the programmed text die* 
well because they had to be self-reliant and not because of the 
• superiority of thq material itself, Tha,s, iftdeed, raises an in- 
terestxng question for future research. If students do well in an 
independent study mode, is it because of the teaching strategy per 
se or because the students possess the quality of self-reliance 
and self-motivation? , 



*, ********* 



Bach, G»L. , "A Further Note on Programmed Learning in Economics,'; 
The Journal of Economic Education . Fall, 1969, Volume 1, Number 1, 
Pp. 56-59. ; " B " . 
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Cffcis study is also briefly described in* the summary of the 
Attiyeh, Bach, \ and Lvgnsden study akove.) " vu ' 

v - } 

Bach reports on a study comparing programmed teaching with 
results obtained by "an experienced, higjhly skilled teacher in. • 
a conventional classroom. . . " which was conducted, at Stanford .Uni- 
versity during one quarter in 1969* The experiment dealt only 
with! microeconomics,^ and the micro portion (Part A) of the Test 
of^Ujgderstaxvding in College Economics (TUCE) was used. About 200 
students were involved. ^ ~- - 

After* three~*introduc tor y classroom sessions^ the students 
jwexa.jallQwecL 'to -.choose fxom* among v the following: * ' 

i ' 1. Read th t e programmed "iiext and do not m attend, classes. 

2. ,Read only the Regular textbook and do hot attend 
classes. . . * 

3. Read cnly the regular textbook andL attend classes. 

Of course,, those choosing 2 or 3 did not have to r read the entire 
book — only*thcoe portions on^the relevant micrbeconomic topic's. 
A week, later the students took an, i&entical examination and were 

^told^.-thoy^would be graded* on three separate curves so that] they- 
would not be penalized for choosing one scheme ever another. About 
qjie third o£ vthe students chose each of the three alternatives. 
The grade- point averages of the three grbups did not differ -sub-*-- 

°stantially. ; - *~ 

The conventionally, taught students (No. 3 above) achieved a 
slightly higher jaedn score on the test than those in the other 
^gtfoups and spent somewhat more time on the course. Yet, Bach 
concluded that "programmed learning appears to b£ a very efficient 
'means of learning /the .core of economic analysis, especially in 
course?, with average or'ljelow^-averagc teachers. ^ (P. 58.) However, 
^ e asserted that with a high-quality teacher in the conventional* 
section "the comparative efficiency of progrcUtuned-learniiig-only 
for students vanishes." (P.* 58.) The students in the conventional 
Section found. the week's work more interesting than did those in 
tjhe programmed- learning section. In summary, there is a place for 
both programmed learning and high-quality classroom teaching', ac- 
cording to Bach. 

• ' \ ■ ' * 

• ********** 



Becker, WHJLianTE., and Michael K* Salemi, The Learning and Cost 
Effectiveness of AVT Supplemented Instruction: Specification and 
Misspecificaliion of Learning Models. Minneapolis: University of 
Minnesota Center for Economic Education, 1976., (Paper for 



presentation at the Midwest Economics Association Meeting", St. 
Louis/ Missouri, April, 197?,) 25pp.. • 

% Becker and Salefoi examine thejself-paced audio-.visual tu- 
torial (AVT) approach^? terms of its impact on quantity, cost 
and ef f iqiency~ of student learning in a community college eco-_ % 
hemic pringiples course- "the following questions are raised:, 

Can a JLearning model be specified , pn the basis of formal 
%* theoretical and statistical grounds * .within whiek learning mm 
can be examined in control' experimental groups?' 

. ^ . • .J > ^ 

' ^Is there a difference Tn the' quantity of learning as 

measured by the TUCE between control and experimental 
c ' ^ groups^? /' , 

*. " 
What influence does student classroom and study time have 

j on learning? . ' # 

Is the learning produced in the' experimental sections less* 
costly to students than that proceed in the control groups? 

Six community colleges in tfro states were. us6d* At each college 
the ^ame instructor taught both a* control group using, his or her 
regular teaching method and an., experxmental group using David A. 
Martin's AVT package^ Introductory Ecoitofoic Theory . Th^Y found _ 
no difference between control and experimental .groups Tn terms of 
gain a scores (post«-TUCE minus pre-TUCE) ♦ Further, student study ^ 
time had little effect, and there was actually a ijega^ve corre- 
lation between pre-TUCE scores .and student learning* There was 
*no difference in the average cost* of learning per TUCE gpint be- 
tween the two groups. 

• One of the simple linear learning models used was as follows;: 



L = f / A, T, S, vj 



L is Learning; A is Aptitude; T is Time input; S is Situation 
(pnysical learning plant); and u is Random trror. The model says 
that learning is dependent (except fox* the error component) on the 
student's aptitude (A is a measure of human capital), time spent 
in class and in study, and the environment or situation in which 
the learning takes place . Such variables as age .and sex were not 
included on the ground that their contribution would be reflected 
in A. m s 

Although the experimental treatment had no. differential ef~ « 
feet on learning, there were significant differences in leafrtfing 



-across schools\ The negative coefficient on pre-TUCE casts (^pubt 
on*pire~TUCE as a raeastire of aptitude. Other researchers may want 
to pay particular attention to Section III of the paper, "Non 
Linear Learning Models and Correction for Simultaneous ^Equations 
Bias." Here Becker and Salami study the .implications of, fitting 
the, learning model. set forth above, and discuss pre-TUCE as an 
aptitude proxy. They suggest that a simultaneous equation bias 
may account for the highly significant negative pre-TUCE coeffi- 
cient estimates. *The»use of an instrumental variable procedure, 
such as two stage a&ast squares (TSLS) is offered .as an appropriate 
remed£. Now the TSLS pre-TUCE coefficient estimate becomes positive 
and this is consistent with the assumption .that pre-TUCE is a proxy 
for. altitude.. Nevertheless, they still find'no "discernible dif- 
ference" between control and experimental group learning; and stu^ 
dent tiirte remains insignificant. 

Next, they take, up the gap closing model . , The "gap" is the 
difference between a perfect score and the student's pre-test 
score. It is the distance the student must close to achieve*, a 
perfect score on the pos^-test. The student's pre-test score is' 
then deducted from his or her post-test score, and this amount is 
divided by the gap, .The result is the percentage of the gap ac- 
tually closed by the student. For example, if a student need3 to^ 
g£in 10 points to close the gap between his pre-test score and a 
perfect score, and he actually gains oniy 6 points, then his gap - 
plosing score is 60%. (For an analysis of the gap closing model 
see Frank W. Gery, "Is There a Ceiling Effect "to the Test of Under- 
standing in College Economics?" in Arthur L. Welsh, Ed., Research 
^ Papers in Economic Education . New Yoxrk: Joint Council on Economic 
.Education, 1972,, pp. 35-49.) With this measure of learning — the 
percentage of the gap closed — there i£ a positive relationship 
between pre-TUCE and student gain. However 4 Becker _and Salemi as- 
sert that the gap closing model is misspecified in at least one way. 
The model predicts that the change, score will be, positive for all 
students, but this fails to account for guessing. Thus, a student 
might score higher" on th<* pre-TUCE than on the post-TUCE.* Beqker 
and Salemi dealt with 1&is by dropping those cas.es for which the 
change scores were negative. The coefficient of pre-TUCE was sig- 
nificantly positive when estimated by. TSLS. They urge that "future 
research . . . strive to collect data which will give information on 
-the aptitude of students in economics to use either as a replace- 
ment for pre-TUCE or as an instrument for it." (P. 17.) 

There was little difference between control and experimental^ 
groups in terms of learning, and student study time was similar for 
both* Thus, the added £Ost of the Martin material might not be 
justified. However, the opportunity cost of student time differed * 



*Several of my own research projects support Becker .and Salemi here. 
— George Dawson. 



because the money value of that time (as measured by the wage - 
rates earned by vorking students) varied. The student^ cost of 
learr*ing was less for the experimental group, and this^ offset, 
all. additional fixed costs of the Martin package. The average . 
weekly student cost of learning per TUCE .point, hbwever f< was not 
statistically different. - ^ \ 

In conclusion, in both the linear and non-linear model speci- 
fications, Martin* s AVT package was -not found to bS superior in 
inqreasing economic learning, and student classroom and study time 
did not prove to be a significant input* The pre-TUCE effect- jovt 
learning was positive when properly estimated by C TSLS. The 
authors do not find the added cost of Martin's material to be jus- 
tified. They assert that their study provides "a sound statistical 
modeling procedure which previously has not been attempted in 
economic • education- " (P. 21.) * * 

(See" the Walstad study, described below, which is closely related 
to this one. The holies and Ginraan \study , summarized b£low, also 
deals with Martin's AVT package. } tt # 



*********** 



Billings, Donald B. , P.S.I, versus the Lecture CQtrrSe in the 
Principles of Economics: A Quasi-Controlled* Experiment. Unpublished 
oaper. Boise: Boise State University, 1974* * 

* At Boise State University, Billings experimented with se^f- 
paced instruction ih a one-quarter introductory macroeconomics^ 
course. An. examination was prepared, using itejas from the Test of 
Understanding in College Economics (TUCE) , and wa^ administered to 
the 31 experimental aad 45 control students. The^ control group 
covered the same material, but the conventional lecture method was 
employed . 

"After controlling for ability and age, Billajbgs found no^sig-' 
nificant difference in performance on the test. He also tried 
several models in an effort to investigate the correlates of achieve 
ment and possible .interactions of student characteristics — class 
standing, sex, ACT scores, math scores/ attitude toward the course, 
and number of units mastered — v/ith self-paced instruction. He 
concluded that the reqression equations offered "few statistically 
significant insights" and that "it appears that all kinds *of stu- 
dents can find it^ a rewarding experience." (As quoted by Elisabeth 
Allison in "Self-Paced Instruction: A Review" in The Jourm 1 of 
Economic Education , Fall 1975, p. 6. Since wd were unable - to obtain 




a copy of Billings' paper, we have relied upoi. Allison's summary 
ror the above, outline of his, study.) . ■ 



Bocfas, Bernard H. ., and D. Lynne Kaltreider, "Computer-Aided Instruc- 
tion for Large Elementary Courses The Am erican Economic Review. 
May, 1974. Pp. 408-413. * 1 — - 

This study deals with the development of *a Computer Generated 
Repeatable Testing system, {CGRT} , as phe means of dealing with* the 
problems presented by -large enrollments. It is similar to Allen ' 
Kelley's TIPS, which is described below. (See Kelley, "Individual- 
ized Education through the Use 6f Technology in Higher Education. 11 
Indeed, the reader may wish to read the summary of Kelley • s study 
.first, since TIPS- predates CGRT.) * » " 

CGRT is a teaching-testing system implemented by computers. 
The tests not only monitor student performance but serve as part 
of the learning process. Administered weekly, the tests (which 
are generated "by computer) are used in conjunction -with tutorial 
services rendered by graduate assistants, .students may repeat 
tests? -and the highest grade is the one that counts. (Note that 
m Kelley's TIPS the tests — or "surveys," as Kelley .calls them — 
are not.-used for grading purposes.) Students schedule their ex- 
,aminations, set\ their awn pace, and receive tutorial assistance when 
they want it. 

" ' - • • " J 

\ Each exam covers the material in the relevant instructional 
unit and. ,xs -prepared by computer' from a question bank of 4000 . 
multiple-choice items. Each test contains 15 questions. The com- 
puter also provides answers, comments, ,and textbook page refer- ' 
ences for each item. Immediate teedback is provided for the 
student "and the grades are recorded. The system is relatively 
expensive t 'however. It was estimated .that each test cost $0,147, 
as compared ^with^ 0.11 for conventional examinations. (Preparation 
costs,' paper, printing, supplies, secretarial' services, processing 
. an'd recording were taken into account . > 

The Test of Understanding in College Economi cs (TUCE) was 
administered on a pre- and post-test ba^sxs. The^ CGRT students 
did 23.5 percent better than the national norm group. (Simple 
gain scores were used here ~v the difference between .ore-test 
mean and post-test mean.) The course also received favorable 
racings from ttu. students, 85 percent saying they would recommend 
it to other Students. Over 82 percent said that the computer for- 
mat was .a good or excellent method of teaching. Nearly 90 percent 



found tjie immediate feedback to be very helplul', and 90 percent 
considered: the repeating of tests to be worthwhile learning 
experiences ♦ . \ 

k ef f<£rt was^made to determine the lasting effects of t£ie 
system by administering the TOCE a year„ later* Over 70 of the 
CGRT students (out of the original 320 at the University of Penn- 
sylvania)., took the test again , revealing that some erosic:. had 
occurred. The oriqinal" gain score* had been 6,87 points. The hew 
gain score was only 3.91 points, a decline of 56.8 percent* {ky 
own computations b#sec$ upon figures in Table 3, p* 412, indicate 
that these students knew 68 percent pf the material on the ~TUGE at 
the erid of the course,, and one year later they still .knew $t> per- 
cent of it,} Eighty-seven percent of the students still considered 
CGRT to.Qe a good excellent method of teaching* one year aft6r 
having been exposed to it. The tutors. were of the opinion that 
they* learned more $but worked harder) under CGRT than they did^ in 
regular recitation sessions* "(See* the Siegfried §xi& Strand study ^ 
below, or* a more systematic analysis of the impact of a similar 
' experience on ' the tutbrsf. } a f . 

The authors conclude that CGRT holds great promise and "that 
it .can be adapted to individual, students 1 needs , even to the point 
of becoming tq tally st\xdeat~paced\ They reedgnize, of course , that 
farther research is needed, • 

(This study is also described in Bernard H. Booms, "An' Experi- 
ment m Computer -Managed Instruction," in George G. Da^sorx, Editor, 
Economic Education £xperiences of Enterprising Teachers , 'Volume 10* 
New york:<Joint Councxl on Economic Education, 1973. Pp. 86*91. 
A more complete report, including test items on computer printrouts, 
details of testing procedures,, cost comparison data, the evaluation 
questionnaire, and statistics on the formal testing can be obtained 
from the Ohio University Library t Economic Education Awards Materials 
: ^hens, 'Ohio, ) . ~ L: . ' 

. ■ i 
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Bostow, Diane, and G. Hartley Mellish, The Effect of Remediation 
on the Performance of Principles of Economics Students Using Pro- 
grammed Hater lals. Unpublished paper* Tampa: University of South 
Florida, 1974* 16pp. 

This study is not the usual comparison of the performance of 
students using programmed materials with those using some, other 
mear*3 of" instruction. Rather, it deals with one asp^dt of many 
courses in which some sort of self -paced instruction is employed*. 



That is< students may often retake . a fi test if they have not 
Achieved a satisfactory score* In a& earlier study , Bostow* 
aind Connor required students to "remediated weekly quizzes when 
scotes fell below 90%. Comparing these students with others who 
could nok take the tests again , they found 'that the ^remediation" 
•group did better on t$e first exam* (See b. Bostow iind R. O^gijnor^ 
"X Comparison of Twq College .Classroom Procedures: Required fterae- 
,d^*tion versus ifo Remediation/ 1 Journa" of Applied Behavior Anal - 
ysis , 1973, pp. -£99-608.) " zss \ 7. . ~ "7 * 

At the University of South Florida in 1974 econcmi^^,st^ents 
using a programmed text- were randomly divided into two group's* . 
Group A students Were required to remediate each weekly test if 
scopes fell l?eloW 90%, Group B coyuld not iretake the te^s, so 
Group A did better. However, ther4 : ,were also differences v in fre-,_ 
quen^y of testing. To conduct a study in which remedia s fcic(n would 
be tne on^y, in4ependent variable to change, the researcn&£s. divided 
ecohdmics classes randomly into two groups X and Y — which 
would experience the same conditions except that X- students could 
remediate during weeks when*Y stud^ftts could. not. Thus, the tech- 
niques of multiple .baseline * aftd reversal were employed. Po'r 
example, X could remediate tests IT 2, and 3, while X could not* *«. 
Then Y could, remediate tests 4, 5, and. 6, but X could not. Refyar- 
S#l, wherein each group experiences both treatments , j^akes^it \ 
possible to have a withih-group comparison o£ the effects of both 
treatments^ The multiple baseline technique reduces the possibil- o 
ity that/accidental variability will affect the research. The 
problem of contamination whith is common in experimental vz. control 
group design^ was avoided* - 

m Students tooJc a multiple ' choice test during the fj.rst class m 
meeting of each, week. ^Those scoring below 90% took the test ag&in 
two days later, plus: a f ew^ sh6rt essay questions. It was thought, 
that students* mj^jht iiot hothe? tp do well on the first test if they 
knefr they would hevd & second chance. This was unfounded/ however. 
The researchers asserted: "...we mu3t assume that students,' initial 
test performance is not affected by the presence of the remedial 
opportunity." (P. .7.) The majority of those who remediated, how- 
ever, did improve thpir knowledge of the material. This occurred 
even when alternate test items were used in place 6f items identi- 
cal with those on the original tests. Indeed", "more students 
improved on the remedial test when the items were completely dif- 
ferent." (P. 6.) The authors conclude that: "There is evidence to 
believe now that students perform better oh remediation for some 
reason other than just postponing the studying they could have done 
for the first test." (P. 8.)' They urge further research to see • 
whether students who have been able to remediate perform better on 
filial exams. M > 



Buckles, Stephen G., §nd Marshall E." \McMahon, " "Further Evidence 
on .the Value of Lecture in Elementary. Economics,? The Journal 
of Economic Education ,, Spring, 1971, Volume 2, Number 2. . 
Pp. 138-141.* • ^ . ^ . % m ' , . m 

Although the .Buckles-McMahpn study d£es*not concentrate on 
personalized, self -paced, pr individualized instruction, It has 
been included here b£c£t*se it did involve the :use of programmed 
.material. Furthermore', personalized instruction .< of ten implies 
the absence of lebtures, and this study may shed some lighten 
whether or hot the abandonment of lectures has serious consequences. 

First, the authors criticize some of tlie techniques used in, 
other studies, l^heyij refer *t9 the studies'by Attiyeh, Bach/ and 
Lumsden, by Bach* alone, and by McCpqjnell and L.amphear, reported' % 
elsewhere" in this booklet.) One "weakness," according to Buckles r 
and McMahon, i$» allowing studehts to select the method of learning - 
(see Bach f£r an Example of this) because they n\ight select 'tech- 
niques under which they learn better and thus bias the results. 
A second weakness is the use of examinations written by the in- 
structors involved, for this enables them to stress material on 
the test in their lectures. 'Further, they do not believe that a 
textbook written, by a teacher involved should be employed. > Finally, 
1 -w-y assert that "only tone^ aspect of teaching should be tested. 
For example, the use of a programmed text etlone should j\ot be com- 
pared to lectures accompanied, by a conventional "text, as the re- 
searcher is not able to isolate, the effects of the conventional 
text ds opposed to the effects of the lectures..," (P. 139.) 

The experiment reported here was conducted during the first 
two weeks of the secon^ semester of a two-semester, introductory 
course, and involved^ two sections taught by different instructors. 
During t f our class periods the .material covered^ by the text was. 
"elucidated. " Then, in each section students Were randomly divi- 
ded and assigned to the control group, (attending lectures and ysin< 
the" programmed text) and the experimental group (not attending qlalls 
but reading the programmed text) . The groups were g?adedtUtt£j3ep^ * . 
arate curves. The micro portion t>£ the TUCE was used as a pre-t^sst, 
with the results used as an independent variable indicating thej!\* 
economic knowledge, each student brought to the course. The post- V 
test was made up' of items provided by the Joint Council on Economic". 
Education from its fc&st' question bank, according to the. specifica- 
tions- of ■ the researchers. The instructors did not see this test 
before it was administered. - • 

Regression, analysis, indicated that lectures did not improve • 
student performance, leading to the^ conclusion that "class lectures 
fthich do no more tftan* recapitulate the "assigned reading do not aid 
the student ix\ learning elementary microeconomic theory." (p. 140- ) 
The number of hours sjpent in studying the programmed text was not 
significant, nor was the se^titon in which the student had been 
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i\ . .placed. TJieTUCE pp,st-t^^t^w&s administered at. the end of, the J 
/jssS^^pster r but this did, ^i;' show that ,'l^c tare's had ad'ded signi£i- 
caintl^ to tlie students * jL^arriiffcJ 6f economics either. Thus, 
lectures did* not have a gj^£er* residual impact (lasting effect) 
; :> \l 'tharij-tha other , mad? of ihs^uctioji^ and McMahdJ^are riot 

r. " pimply ing^fej^at .lepti^s^^hould bj5 eliit^natdd, however. Note "f:Jb£ , 
qualifying statement s " lectures which da na^mQre than recapitulate 
v the "assigned reading. " 'They Believe, rather, that lectures might 
' >rw be* replaced,^ discussions of applications ofl economic theory. 

They recognize, too, that more" research is needed, b.ecause their , 
^ ydata* applied pniy'to abemt 160 students' in one university (Vander- 
t\ ~ £>ilt) ♦ .in their concluding comments £h£y state that "it would be 
jguj^ of immense vaiue to determine what kinds pt students do best . under 
^"\* * which alternative teachihg methods. Perhaps then education could 
be individualized at even the largest universities." (P. 141.) 



TO 
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Calvin, Allen D. , "Programmed Instruction in Economics," in 'Keith 
G. liumsden, Ed., New Developments in THte "Teaching of Economics 
'(Englewood Cliffs, N.J.: Prentice-Hall, 1967), pp. 59-64., 

i * . . , ' ■' / ' * * ' 

, > Calvin reports on a study conducted by R. Bryce Young at 
FopthjLll Junior College in Los Altos Hills, * California. The col- 
lege offered, a course calicd "Business Economics" which attempted 
to coyer in one sem^s^ter the areas covered in the conventional two- 
semester .univer^fcy course in principles of economics, A group, 
using a standard textbook and getting conventional instruction was 
compared yith ariafcfrer gfoup using programmed material developed by 
**A£fciyeh',, Lumsdefi;^ i^id Weiner ( The American Economics Series pub~_ 
/Tished'by Behavipral\Rese^rch Laboratories of Palo Alto, California, 
in 1963 and 1964J . \' + -. ^ s 

The experimental g^oup did not hear lectures, but did attend 
regularly scheduled classes to take review tests and discuss indi- 
vidual problems with the instructor,. "There was no significant 
< di£fQ£ence between tfte^o groups in terms of median scores oi) the 
American College TegtinG^g^ogr^a Entrance Examination (Chicago* 
Science Research As'sd^ates, 1964J . Science Research Associates 1 
T&st of Economic Understanding wa£ used for pre-, and gost-testing. 
The mean improvement of the experimental group was more than double, 
that. of the group using the conventional text. According' to Calvin! 
** ,r An* analysis- of variance indicated that the improvement was signi- 
JEicant j»4 attributable (P r 63*> 

Furthermore/ the group using the p^rogr^nmea ^terials did signifi- 
cantly better t^an the national norm group for the.TEU. Finally, 
the cont^ITgroup s3id no better on the instructor * sj own final 
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examihat ion than did the experimental group* Thus, Calvin* con- 
cluded/ that /"programmed textbooks can be an extremely valuable 

/tool in/the teaching of economics.." (P. 630 1 

* ♦»> ^ 

mm **h*» • . 

o * * * * * * * * Tftr * 

* Craig, Eleanor ^u, ?rvd Jamesr B. ^O'Neill, The Advantages of the 
.Humanistic App^a&h in Teaching College Economics (Newark, Dela- 
ware:. The Bureau o$ Economic arid Business Research, College of 
Business and Economics, University of Delaware, 1974), 17pp. 

/ After compar.ing r sever'al course formats in introductory macro- ' 
economics, Craig and O'Neill concluded that "The more that the 
student w^s involved in his own learning .process through self- 
instructional techniques and individual options for prbject3, the 
moz^.he perceived that he learned, the more he enjoyed the in- 
structor, and the more favorable changes occurred in his attitudes 
^toward the subject matter." Five classes of about 30 students e^ch 
• N were, tested with the TUCE and - a departmental fin^l, and .filled out 
a^ttitudinal questionnaires. Two classes were taught by "Instructor 
-A/' who used' tlie. Sterling Institute's Economic Analysis programmed 
materials, Volumes 1, 2, and 3.* Current readings were al£o as- 
signed, and £here wasi ^frequent testing." Instructors B^pd C used 
a autre traditional textbook , handouts, and supplementary' readings. , 
Optional independent activities included a computer game, trips 'to^ 
Washington and New York, debates, book reports, case studies, papers 
and ofc^er indiv-idual projects.^ v v : 

\ *** * - « • * 

.The TUCE was used as pre-test and post-test' in all classfes, 
revealing no signif ic^aht difference in gain scores. There was no 
significant difference in performance on- tfce departmental final, 
and there was a high correlation between performance on that test 
and on the TUCtfr- (The students were found to be similar ih terras t 
of SAT math and verbal 6cor&£, and in .terms of * grade pointy indexes.) 

Significant differences werfe found in^attitudinal response* 
Students v;ho had optioiis fur outside activities appear to have de- 
veloped "a keener' interest in the discipline." (P. 7.)' The authors, 
assert: "The more positive attitudes throughout \he semester for 
instructor's B and C seems to have been generated by t)ie greater 
student Involvement in the learning process. V{P. 12 O * 



******* *-*f:* 

^For -other studies of 'the Sterling materials, see Mellish and 
Bostow below , and Soper and Thorton below. 
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\Craig, .Sleanor D., and James B. 0 ! Weill , -"The Predictability, of 
Retention in Economics," The Jou rnal of Economic Education , 
Spring, 1976, Volume 7, Number - 2. Pp. 92-94. 

.Students taught by conventional methods ta standard intro- 
ductory macroeconomics textbook plus a book of case studies} were 
compared with those using self- instructional materials. Again, 
the TUCE, a departmental final, and an attitudinal questionnaire 
wer& used, (See Craig ; and O'.Ne^ill, above.) Although they had k 
found no "cognitive level differences," Craig and O'Neill thought 
that students who had mot e positive attitudes toward, the course 
might retain the material longer. Their hypothesis was t confirmed. 

The population tested included students who had taken ^radij 
tional courses and self-instruction courses two years earlier and 
one year earlier. Some students had since taken additional ^eco- 
nomics courses, and the jnean score qn the "Retention TUCE", was 17*7 
as compared with ,15 . 6. for those who had taken only the introductory 
course. A factor summarizing "positive attitudes toward the course 
explained eight percent of the variability in retention. The sel£- 
instructional method accounted for another twc percent, but this 
was statistically significant. 

* .» 

The TUCE items are categorized as "Recognition and Under- ' 
. stemding" question^ , "Simp3:e Application" Sterns, and "Complex Ap- 
plication" items — .the flatter being the most difficult. Craig .J 
\arid*6*Neiil found n6 significant differences in retention between * • 
"^he self- instructional and traditional classes v in any of these oog-j 
nifcive areas for all students in the sample. / However, with students 
who had taken the introductory course only, there was a relationship 
between ,time elapsed since .the course was taken ancTthe,, method of 
instruction. $6th methods produced the same level of retention for 
students, who had taken the course one year earlier. But for studerts 
who had taken the course two years earlier, the mean retention on f 
Complex Application items' was 16 percent higher for those who had 
V&eri exposed to the self -instructional method. [See AttiyeK, Bach, 
and Lunisden,. above, £o£ a similar finding.) Craig and O'Neill cojt- 1 
, eluded that* ^people remember things whicjh interest them. " r They * * . 
"assert thatV^When post-TUCE performance and verbal ability J4erfi_ 1 



"discounted, attitudinal interest was the next most important vari- 
able in predicting retention results." .{P. 94.) 
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Darnton, Donald C*, "Programmed Learning-Policy Analysis An 

.Experiment in Teaching Princip 1 es of Economics , * The Journal of 

Economic E ducati ons Fall, 1971, Volume 3, Number 1. Pp. 32-35* 
- — t . s . 

Virginia Eblytechnig institute participated in the nationwide ' 
^tudy of programmed learning conducted by Attiyeh, Bach,, and 
Lumsden (described above) , Seeing that their own results agreed 
wxth the nationwide findings, vpj decided on -further experimenta- 
tion with programmed instruction. A group of 42 students in a 
three-credit introductory macroeconomics course made up the experij 
mental section. They did not attend regular, classes, but met to 
take weekly tests (multiple-choice and essaif),, and; could meet with 
Darnton on an individual basis to discuss their progress, 

Darnton asserts that the "substance of macroecdnomics" was 
coverred in four ,weeks, after wnich the students worked on "position " , 
papers involving different economic -policy issues. The experimental 
students* reportedly devoted less time to the course, as did the 
instructor • Part One of the fcilCE was administered on a protest, 
post-test basis to the experimental and two control sections, with 
the. result that the, experimental students did jus£ as well as those 
in the control groups -and in the national norm gr6up. They did 
better, however/ on the complex application type of questions, while 
" the controls and the norm group did, best on the recognition ques- 
tions. Darnton explains this by spying: "Conventional texts con- 
tain far more descriptive and institutiqnal material than do pro?- 
grammed books , "and this information is iSetter tested by questions 
requiring recognition than application. The policy, analysis papers 
written in k the experimental section should have helped students to 
apply the theory they had learned , and. so probably prepared them * 
for those questions requiring znore complex application of analytical 
tools. 1 *. (P. 33.) 

The method jproved to H be Jpop,ulctr with the .experimental students 
(ail of whom had~ previously been epcposed to conventional instruc- 
tion in economics) their evaluations being "Overwhelmingly favor- 
able*" (P. 33*) While Darnton does not suggest th&t the programmed 
text; plus policy analysis approach replace all others, he does ar- 
gue that his experiment confirms "that it may.be one of several 
viabXe methods of teaching principles of economics" (p. 33) und that, 
it has ^sufficient nierit to add it to the variety* of teaching methods 
used in the principles course." 1 (P. 34.) 
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T . n --„ Hendigs, "The Vanderbilt-JCEE Experimental CojArse in 
s Elementary Economies , M Thg Journal of Economic Emicfltion. 
J.£p6cial'Issu6 No. 2, Winter, 1974 . 94 pp. ' ^ p\ 
-\f : .; * • • * Vy , 

„ t ; The experimental course at Vanderbilt involved £he adaptation 
*! ;of. the case fc^thod to economics instruction* Several years were 
; "spent in developing the course and testing* its effects . An at- 
j tempt was made to promote student mastery of econ<Stnic principles 
t through a combination of the "case me^tiod with Keller 1 s jgelf-paced 
t P.S.I. In place of lectures, the students get an assignment which 
f they study with tjhe help of a proctor , then take attest. when they 
think they are ready, for it. Complete mastery is expected before 
the student goes on to the next assignment, and grades depend upon 
. the .number of assignments completed. Students know what they have 
j-feo do to achieve an A, B, or C grade,, and incentive is high. Stu-r 
} deats work at their own pac4, having personal contact with instruc- 
tors when needed* Students may earn up to 80 points during the 
lemfester and up to 20 more on the final exam* .One must earn 90 
points fc*r <an\A, 80 for a B, etc. * 



For. each jof seyen sections of 'the principles, course there is 
7 a written test on. basic concepts* .This is worth four points. The 
concepts are those which the student must know in order to analyze 
economic issues and cases., — The seven tests ^are in the study guides 
and students must pass each with 100 percent correct. There is also 
-an analytic test for each of the seven sect^ops, also worth four 
points. Sample questions , but not the actual^ test items, are avail- 
Able' to students in 'adva$c£* Students may take, these tests~ either 
"ih vfritten or oral form. Th£ latter iorm is considered' w a powerful 
i method of teaching. " (P. 7,} Students earn 56 points by passing 
the basic concepts^ and abstract analysis tests, and they must do this 
to pa^s the course* , ~ 



, Students also earn points by passing tests on policy cases. 
(See Hendigs Fels and Robert G. Uhier, Editors/ Casebook of Economic 
Problems" and^Policies . St* Paul: West Pubi. Co*., 1974,75.) They 
study a .case (such as the economics of an all-volunteer army), de-^ 
velop their own positions on the issue, write it up, and have their 
essay evaluated-. If the essay is. deemed "excellent, n the student 
goes, on to a" new case. . They may fearn up, to 24 points in this manner- 

Departing spmewhat from the usual P.S.I, procedures, Fels re- 
tained a conventional final examination. He believed that by re- 
viewing for the final' the students would reinforce their learnings 
and perhaps retain the material" longer. 

• * * 

Preliminary evaluation, while limited, was/f avorable enough to 
encoutage continuation of the experiment* The results are set forth 
in the study by* Siegfried and Strand,, described belowJ (Pp, 9-26.) 
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(If the preliminary findings are of interest, or if readers wijsh ^ 
to exsimine other related studied by Pels, see the following: 

Ben Bolch, Rendigs Fels, and Robert Uhler,. An Experiment 
with Self-Paced Instruction. Unpublished paper . Jflashville; 
Vanderbilt University, i972. (Self-paced instruction #as / 
used in a basic Macro course in conj.unction with a pro- 
grammed text* The performance of the 27 experimental stu- 
dents was compared with that of 109 control students on the 
TUCE and the CLEP. No significant difference was fojind.) 

Rendigs Fels and Dennis R. % S tar leaf , "Controlled Experiments 
in Teaching Techniques." SQUthern Economic, Jour Aai . July, 
1963/ Pp. 68-73. 'J 

Rendigs Fels, "Developing Independent Problem-Solving Ability 
in Elementary Economics." 7 American Economic/Review. May? 
1S74. -Pp. -4*03-407, ' , V 

• * » 



Fusfeld, Daniel R. , ahd Gregory Jump, "An Experiment with Program- 
med Instruction in Economics*" The Southern Economic Journal . 
January, 1966. Pp. 353-35&K . 4 

In one of the earliest experiments with programmed instruction; 
FUsfeld and Jump at the University of Michigan evaluated the effect 
of programmed materials, on learning supply and demand. The ma- 
terials were used, during, the first week of the s.econd semester of 
the introductory "principles course. The normal procedure was to 
have students attend £ one-hour lecttfre and three one-hour discussion 
sessions each week. In this experiment, two discission sections were 
excused from classes far one week and were toid. to study entirely on 
their own, using programmed material. The same test was used both 
for pr^- and post-testing. Students kept logs on the time spent in 
studying * 

The two control sections were taught by the same instructors 
responsible for, the experimental sections. Students were told they 
were part of an' experiment and knew they would be tested after one 
week. (The test was made up of 20 true-false arid five completion 
items taken front the programmed material and the workbook used in 
the, course.) The post-test was to dount'as part of the students 1 
final grade. The basxe questions posed by Jbhe researchers were: 

/ 

/ 



/ 



(1) Can students learn as much froiriHndependent study using 
~ . .Rjrogrammed materials as from conventional instruction? 

. (2)" »e£'the textbook make a difference when used in con- , % 
* v^i junStibn with programmed material? • . . f 

(3) Is learning improved when students are aware, that an 
t experiment is being conducted., and that they will be 

~~ . i 'tested? * " 

(4) Does t independent study save time for students and/or 
teachers? 

- The pre-tfcsts revealed no significant differences between 
groups* The post-test likewise showed no difference , indicating 
that programmed instruction was as effective as the conventional 
method* When used' in conjunction with programmed material , #t the 
textbook added nothing to student achievement. Student awareness 
_o£ ,the f act that they were part of an experiment and that they » 
would be tested did make a significant difference in their learning^ 
(Two '"secondary control sections' 5 had~*been set up, consisting, of 
students who were not told* they were part of an experiment and were 
not tcrld that a test would be given at the end of the week.) 



The data were 3l'so broken down- by bright, average, and poor 
students. The lower third shtowed the largest improvement; the top 
third showed the leasts Thrs was to be expected on a test with an. 
upper limit. More importantly, however, when the bright, average,'* 
^ahd poor students were compared across groups (experimental ahd 
control) it was revealed that the programmed students did as well 
,aft their peers in the control sections. " % / 

Finally, the programmed students sjfent less time in studying^ 
but learned as much, as their colleagues in the conventional sections. 
Fjjsfeld and Jump recognized the fact that they were dealing with 
smair samples (about 70. students altogether)^, but concluded that in- 
dependent study can bfe as effective as converjtional instruction while 
saving time f or .both students &nd instructors. < 



t *********** 
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Gordon/ Sanford, "Optimizing the Use of Televised Instruction, * 
The Journal of Economic Education . Fall, 1969, Volume I, Number 1. 
_Pp. 4 6-50 t - : ' . 

o 

Gordon reports on an experiment in whi9h three modes of in- , 
struction were compared at the State University of New York at 
Onecnta. In addition to a conventional lefcture-discussion section, 
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one group of 25 students received basic economics instruction 
through television , followed by ten or fifteen minute discussions 
led* by a senior student* A third group of 25 student.s. worked 
individually in study carrels , using 8mm .film cartridges and set- 
ting their own pace. Students in the differeRt groups were matched 
in ,terms of regents scholarship exam scores", sex, and academic 
major. On the midterm and final examinations, 14 of 'the students 
in the individual study group achieved higher scores than their 
'counterparts, while 11 in. the. TV group had higher scores. The 
differences were not statistically significant, however. Gordon 
noted that further research was needed, but concluded: "If our 
product, the introductory course, is to meet the needs of our con- \ 
sumers, the students, we' must recognize their differences and vary 
our offering to meet their changing' demands. n (P* 50.*) • 



Hanni, Eila,*The Grade Incentive as a Systematic Variable in the 
Comparison of Conventional vs. Behavior Managed Learning of Princi- 
ples of Economics. Unpublished paper. Tampa: University of South 
Florida, 1975. Pp. 12\ 

» 

0 This study involves a comparison of programmed learning and 
the conventional lecture-discussion approach at the University of 
South Florida. (See the Mellish-Bostow study, summarized below. } % 
Hanni hypothesized that*-the incentive offered to students taking* 
the test was more important than differences in teaching m et h od s. 

• * . 

In the experimental group^ the students were tested "at least 
on.ce a week and were required to re-take tests .Klieg. performance 

_#as- below 'taastery level. This group rarely attended lectures. The 
students received behavioral objectives at the beginning of the 
quarter, and their grades depended upon a point system. The control 
group students were tested infrequently, were not allowed remedia- 
tion, and attendpd fc ,l lecture sessions per week, often accompanied 
* by dlcss discussion. These students received a conventional sylla- 
bus and were graded on a curve, A multiple ct Dice examination was 
administered at the end of the quarter to both groups. Most of the 

■ questions were 4:aken from the TUCE. 

There was no significant difference between the experimental 
and control groups. That is, the programmed approach was as gocd 
as — but no better .than — th£ conventional -method.* It doeg make 
a difference^ however, if the test used to compare groups counts . 

-ward the student 1 5 course grade. Hanni notes that "Noras have 
been developed without motivation for the test being explicitly 



recognized as a contributory fi£tor In major undertakings, such 

as the. preparation of the TUGE tests, no uniform motivational frame- 
worjc was set up*". (P. f '5.) This is an important observation/ The 
Manual for the.TUCE points out (p. 13) that about one fourth of the 
instructors administering >the TUCE to collect norm data used the 
test as part of their final examinations and that tfre effect of this 
on the scores is not known. The instructions issued, to those parti- 
cipating in the norming of the^TUC£L_V£ere to tell the students that 
their scores woulS be used, uo develop national comparisons and that 
their results would be compared with those from other schools. Hanni 
argues that "more meaningful comparisons of student performance in 
conventional vs b.ehavibr managed learning situations are obtained if 1 
standardized, validated tests are used in both cases and the contri- 
bution of this test to the overall grade is employed. as an independent 
variable, " Harini^s regre^ssional analysis showed that the, 

weight assigned to the test used 'to compare experimental and control 
groups was a significant determinant of the student's final test 
score. The basic "conclusion is that "there is a need to develop 
testing 'situations, and especially norming situations, where a grade 
incentive is present and controlled for. Norming situations that 
rely on the willingness of the students to work for the benefit 6f 
'future generations of students' defeat their purpose. ...If "a 
considerable portion of the course grads depends on the result of the 
final test, student performance will be good; however, if the test 
Will have little or no effect on the course grade, student perfor- 
* mance will drop 6ff. n (P. 12.) . 



***'***.**** 



Havrilesky, Thomas, "A Test of the Effectiveness of Teaching Money 
and Banking by Programmed Instruction," The Journal of Economic 
Education. Spring, 1971, Volume 2, Number 2. Pp. 151-154. 

Havrilesky f s study differs from most others in that it deals 
with money and banking rather than the^ introductory mic£o or macro 
course as a whole. Tke programs^ text used deals .with money and 
banking, whereas most other experiments used programmed materials 
^n basic principles of economics. ' " ' 



The 36 students involved in the study were taking a macro 
course at Duke University. All were sophomores, and all had had 
ten weeks of conventional instruction which excluded monetary theory 
and money and banking. The experimental group of 15 students was 
required to do ll^short chapters of programmed instruction during a 
4 one-week period. The 21 control students received conventional in- 
struction, attending three class lectures^an<L one individual 

* 



'tutorial session with the instructor. They read several^ chapters 
in "standard textbooks . 

Each group took a pre-test made up of 12 questions an mone- 
tary economics taken from the TUCE jauifl four items prepared by the 
instructor. (The instructor was the same^for both gyoups,) Each 
group was graded on" a separate curv^. "Th^rs was* no significant 
difference between groups in terms of pre-test ntean, grade point 
average/ SAT total score, cumulative class grade, and hours .devoted 
to study during^s^ae week* The programmed group achieved a higher * 
mean score on the ppst-test, however., and the difference was signi- 
ficant at the .005 'level. An. ordinary least squares regression 
analysis revealed that the use of programmed instruction, grade point 
average, ajnd cumulative grade in class were significant at the ♦OS 
level. The use of programmed instruction increased the post-test 
grade by 8.7 percent. 

Havnlesky suggests the possibility that programmed instruc- 
tion has a comparative advantage for. short periods, but that^the 
advantage might decline as the time period in which it is used in- 
creases. Programmed instruction might generate self-reliance and 
cause students to work harder, and jperhaps they "generally work wej.1^ 
with programmed 'materials. " (P. 154.) Contrary 'to some other re- 
searchers, HavriJLesky thinks that programmed learning works well with 
the better students, but he asserts that all students in his study 
benefited. Finally, he concludes that "programmed instruction excels , 
for core learning , aver, equal , short periods c£ application." (P. 154.) 



.... ^ 

Kelley, Allen C, ^Indivj^dual^^ng. Education through the Use of 
Technology in Higher Education," The Journal of Economic Education . 
Spring, 197 3, Volume 4, Number 2. Pp. 77-89. 

While teaching the introductory economics course at the Univer- 
sity of Wisconsin (Madison) , Kelley became concerned at the "deper- 
sonalization" caused by large enrollments. Thus, he developed his 
Teaching information Processing System (TIPS) as a means of providing ; 
some sort of individualization within the context of Targe lecture 
courses. The problems are well known : — professors go too fast for 
some students, too slow for others. Frequent feedback is laefking, 
*and when students do get some sort of indication of how well they are 
doing it may be too : late to Correct deficiencies easily, j 

- Kelley 1 s TIPS combined the results. of weekly muitiple-choTce 
surveys (he declines to use the term "exam") with data about the \ 



students , such as his or her aptitude and achievement test scores , 
math background, and previous performance, Student^strengths and 
weaknesses are identified and Matched to alternative assignments 
"which are reported on a Student Report Form. The student receives 
an assignment tailored to his or her instructional needs.. For ex- 
ample, a, student might 'receive an optional assignment on a concept 
already understood and a required assignment relating, to something 
not well understood. A student wit£ 4 strong afeth -background might 
get an assignment that would enable that individual to utilize that 
background on the economic topics. Students who do^very well may 
get special assignments , such as attending a special discussion sec- 
tion or reading material on a current economic issue. The! surveys 
will also indicate whether or not .a student needs tutorial assistance. 

The professor also gets feedback on the class as a whole, en- . ^ 
ahling him or her to concentrate on weaknesses. How is all of this 
done? Since the professor carmc\t Know a lar g e u mnfogr~o£. students 
personally, a computer processes the surveys and generates the spe- 
cific assignments. The surveys are not used for grading the students, 
hehge Kellfey's reluctance to call them tests. The following examples, 
.will illustrate the use of the surveys: 

(1) Mzf*. W. achieves a perfect score on a surveys Instead of the 
regular class assignment, he receives a special handout, which 
goes beyond the regular course presentations* He is also in- 
vited to attend an intermediate course session*. All of this 
is explained in the computer print-out which reports to Mr. W. 
the results of the survey. The print-out even contains a "touch 
of humor — "Congratulations — you obtained h perfect score. 
Yeu. are either extremely bright, or just plain Plucky. * (P. 85.) 

(2) Mr. J. did not do as well as Mr^ WV He missed two of the ten 
questions. The^computer tells him which items he missed and 
tells him where to look for the correct answers. *He is also 
told to attend a special discussion session. The computer 
"knows" that Mr. J. has a strong background in ipath, so he is * 
assigned to a section m which mathematical techniques will be , 
used in developing the economic concepts. 

(3) Poor >ir. 7. answers only three of the cen items correctly. He - 
1_ is thrtn pzxwidgd^WAlhspecial muteri&J^which are simpler than 

the basic textbook, aK3^i^is told how to learn the concepts 
he missed through the study ^programmed textbook. 

•The system is fast and efficient. The surveys take, five to ten 
minutes to complete, and the students receive their Report Forms 
within four to ten hours > The professor and the teaching assistants 
also receive Report Forms. Thus, the teaching assistants quickly 





learn which concepts are being grasped by their students and -which 
need reinforcement* ine professors get the same kind of feedback 
for the class as a whole. 

.* * 

SSyerai ^Valuations of TIPS hav^e been made. Students in TIPS 
sections were the experlmentals;, students getting conventional 
lecture-discussion instruction served as control^; Those in the 
control group jceceiveu identical assignments, while TIPS students 
r^pfcived assignments geared to. their problems , backgrounds , and ^ , 
abilities. There *ere about 250 students in eacl) gribup. Identic^? 
texts were used, and the two groups proved to be similar in tetfos 
of aptitude, prior academic achievement, academic major, . class, and 
math background. Thai same, inid- term eaair^was administered to both 
groups {at the same hour) . • 

"Through regression analysis, witk controls for such variables 
.as class, major, ACT and SAT scores, pre~t$st scores (using the 
TUCE), sex, math background, .and section leader, it was found that . 
..TIPS increased examination scores by about 15 percent for th§ aver- 
age .student. The benefits of TIPS were not evenly distributed 
however. Low achievers ^ {as insured by' ACT SA&) gained" most 
t^vk^Xta. th$ir test scores increasing by 19 percent. High achievers 
gained, &£out 13 par^rentr. Thus, while all derived benefit from TIPS,, 
, its greatest impacts was op the poorer student* 

Kelley also did a lasting-effects study, retesting students 
after on# year. This revealed that the TIPS students maintained 
their superiority over titae, although its magnitude diminished* 
The proportion of TIPS students selecting economics as a major was 
23 percent higher than that of the controls* Yet, the TlPS^group 
did not g^v* the course or the instructor a higher rating, nor did 
they enjoy the course more than the control students. Kelley feels 
.that further evaluations are needed, although TIPS is now being used 
in 70 different universities and informal reactions are highly 
favorable. • " • " . 

For other reports on TIPS see the 1 following t 

" Herbert^* ; Grui^l m& AXien C* Kelley, TIPS and the #se of 
Computer^, tn Teaching Economics; The Experience of Simon 
Prase , Unpublished paper. Durham, North Carolina t Duke 
Ohiversifcy, 1974. 

Allen C* Kelley, "An Eap**rimenfc with TiPSs A, Computer- Aided 
Instructional "System for Undergraduate Eduction, * A merican 
Economic Review . Hay, 1968* Pp. 446-457* 
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Allen C. Kelley/ *The Economics of Teaching: The Role of 

TIPS," in Keith 6. Lumsdeni Editor , Reeent Research in 
Eco nomic Education' . (Ehglewood Cliffs , N. J. : " Prentice-Hall-, 
B7b.) pp. w > 

Alien C. Kelley, "On the Appraisal of Technical Change in 
* Classroom. Instruction," in George G. Dawson, Editor, Economic 
' Education Experiences of Enterprising Teachers , VciumFT^t 

(New York i Joint Council on Economic Education, 1973.) 

Sp. 78-35>. ' t - 

Allen C. Kelley, "TIPS and Technical Change in Classroom In- 
struction," American Economic Review . May, I97&. Pp. 422-428. 
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Kelley, Allen C, and Caroline Swartz, "Student to Student Tutorino 
irt'.Economi^s,j^ ghe Journal of Economic Education . Fall, 1976, 
'Volume* 8/ Number I. " • ! ^"T * 

* ■ . -J ■ - " • <it 

Kelley* s TIPS, in which computers are used to help" individual- 
ise instruction, is described in the report -immediately above. 
This study reports on "an experiment in which student-to-studen6 
tutoring was added to TIPS in the freshman economics course at Duke , 
University, Students who performed well on the TlPS surveys were 
offered exemption from an examination if they would tutor students 
who were having difficulty. The tutors had to attend a one-hour 
training session,^ hold group tutorials for three to six students, 
and be available for one-to-one tutoring. 

Students having problems were invited/ but not required, to 
attend tutorial sessions. About 18 perqent of the class was so in- 
vited, and 11 percent did attend. Such self -selection raises ques- 
tions about the extent t$ which the experiment was wail-controlled , 
but a comparison, of "background attributes" — SAT scores, high 
school GPA, math background, and exposure to high school economics — 
showed the students who rejected the invitation to be similar to 
those who accepted. Self -selection, then, probably didn't bias, the 
results, unless' self-selection indicates motivation. Kelley felt • 
that self-selection very possibly did indicate motivation, and U»at 
this factor in combination with the tutorials explained _the superior 
performance of the studen^s^^^ accepted tutoring . 

All students took two mid- terras and a f^nal, and, were required 
to hand in five cases. Points earned on the tests and through the 
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cases were added to get total points foe the. course. All of the, * 
^badtground -at f H hn»-f» q " hflfl a*s^ ijtj.cjally signif icant effect' , 
on performance. . The impact of the . tut«5rial. program was much 
greater, however, often raising student scores sufficiently to 
increase their -course evaluations by a full letter grade.. Further- 
more,, the, cost ofcKelley's tutorial program was low. (F<jr the 
effect of tutoring on the- economic learning of the .tutor, see the 
Siegfried and Strand study summarized below. J a 



Luker,. William; Marianne Bonds; Watt Black? and ? Thomas C. Holland, 
"The fcuasiexperiraental Study of the Differential Impact of. Four 
Delivery Systems on Economic Understanding, H The Journal of Economic 

Edu cation . Spring 19 7 5, Volume 6, Number 2. Pp. 134-137. 

— , . ^ * 

The researchers .'measured the impact of four "delivery systems*' „ 
on student«5.nowledge of economics. The TUCE was employed to. test 
the effect of games and simulations, closed-circuit television (the 
taping and playback of student presentations) , programed learning, 
and the conventional lecture-discussion* approach. Over 100 stiraents 
enrolledxin a microeconomics course at North Texas State University 
were tested.. They .were randomly assigned to various groups. The 
students also completed questionnaires on how much theV enjoyed each 
delivery, system, and how effective it was. in preparing them for 
examinations. 

A tutor was assigned to each group with instructions to use 
each delivery system one time. The students were tested before and 
after exposure to each system. The research design was a modifica- 
tion of Campbell and Stanley's quagiexper imental , counterbalanced 
design Ko. U. (See Donald T. Campbell and Julian C. Stanley, 
Experimental and Quasi-Experimental Designs for Research , 8th ed. 
Chicago : Rand McNaTIy , [ 1572 . > 1 * ,•* 

Our interest in this study stems from the fact that programmed 
materials were included, and programmed materials are often used. in 
self-paced instructional programs. The researchers found that 
"There was. no significant differential impact attributable to de- 
livery systems on increased economic understanding." (P. 136.) They 
concluded, however,, that "programmed learning is' probably more ef- 
fective if students recognize its value." (P* 1.3.7>) They did find 
"a significant interactive relationship between increased economic 
. understanding and the degree to which students using programmed 
materials perceive programmed learning as worthwhile in preparing 
for. -tests." (P. 136.. J On the other hand, Jprogrammed learning was 
not as popular as the other systems. The 'order of preference was 



glmes and: simulations, (2) lecture-discussion, '(3) closed- 
circuit TV, and (4) programmed learning. A final cpnclusiop was 
that a multiple set of deli very^-Sys terns can be used, "with the 
most effective combination being programmed , learning and games 
and simulations. n (P. 137.). 
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ZiUtasden* Keith, "The Effectiveness of Programmed Learning" in Ele- 
mentary Economics," American, Economic Review , &ay, 1967. Pp. 652-659. 

* 

7 Lums#en reports on several studies measuring the effectiveness 
of programmed learning ip microeconomics. These involved students 
.at Yale, Stanford, the ~ University of California at San Diego, Wes- 
ley an, the University pf Illinois, and Marin College. v 

„ * *£he first two* experiments involved about 100 students at 
Stanford who had not had economics before in college and about 130 
'sophomores' at S$n Diego who had taken introductory mccro. The 
classes were split into three matched groups called A, g, and Q. 
Students in A received programmed material arid were told to ase this 
exclusively. # Thpse in B were taught in the conventional manner, 
using their regular textbooks and attending lectures covering the 
same mat&rial. The C group studied the regular basic textbooks only 
(the same as those used by B) . All students could attend qa«ustion " 
sessions and consult the professor and the teaching assistants* At 
the end of one month all took the same examination, made up of true- 
false and essay questions. Th£ group A students achieved signifi- 
cantly higher scores. (There was no. difference on .the objective 
part of the- examination, however.) All students then received con- 
ventiona) .instruction and a common examination at the eiid of the 
course. Group A students again outscored B and C on the final ex- 
amination. tums4en could not be sure, however, whether programmed 
instruction accounted for group A*s superior performance .or they 
were superior to be^in with. ' * 

In. a slightly different experiment at the four other schools, 
sections were Randomly selected from the introductory classes. The 
experimental group studied the programmed text only; the controls 
used conventional texts and attended classes. All took the same 
test after four or five weeks. The test was made up of multiple- 
choice items from workbooks. A simply regression analysis was made, 
with test score as the dependent variable and treatment ^group and 
intellectual capacity as independent variables. At Marin College 
the control students did significantly better, but at the other three 
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colleges there was no diff ererfcre between groups. (Marin is a 
-junidr college*} When adjustments were made for time, spent 
studying there was no significant difference between the pro- 
grammed and non-programraed sections. Lumsdert cautiously con- 
cluded that independent s.tudy through programmed instruction can 
be as effective as "conventional teaching (at least for learning 
pr£ce theory) and that it. s^yes time for both. teacher. and student. 
,The programmed sections spent much less time in. studying (less, 
than half as much $t the University of Illinois) > but learped as 
much. Finally , of the 500 students who responded to questionnaires 
on their preferences , 49 percent, chose programmed* instruction, 26 
percent selected the conventional, and the rest were indifferent*. 

(Also see the study by Attiyeh, Bach, and Lumsden, "summarized^ 
above.) ' * ' * r* 

********** 



McConnell, Campbell R.^ and Charles Lamphear* "Teaching principles 
of Economics Without Lectures , " The Journal of Economic Education . 
Pall, 1969, Volume 1,* Number 1.* Pp. 20-32, 

This is one o£ the early studies designed to answer the quesr- 
tion: "Can today's college student. .grasp the principles of eco- 
nomics without the aid of the classroom teacher?" (P. 20.) The 
authors compared the performance of students studying economics by 
closed-circuit televised lectures with a "lectureless" group. Over 
40t> students *t the University of Nebraska were involved. 'Students 
in the first semester of the principles course were permitted' to 
choose between the two groups, and 86 chose the lectureless section 
The Omnibus Personality Inventory (OPI) was administered, to both , 
groups to see if those seleqtihg the lectureless section had more * 
autonomy or initiative. The OPI revealed no significant difference 
between them. Students in both groups "used a 1 conventional textbook 
gtudy guides, readings on current issues, and programmed material. 
The TV lectures were textbook-oriented. Ten tutorial sessions were 
held. for each group under the direction of graduate assistants-. 
Short se,lf-tests were also prepared', enabling students to check 
their progress frequently. 

To compare the performance of the two groups, several tests 
were administered. 0n<* Was a 170-item examination designed es- 
pecially for the Nebraska principles course. A second was the 
Test of Economic Understanding (TEU) , designed for high school 
students but widely used in coll ges before the development of the 
TUCE. A third was a 90-item test made up of questions supplied by 
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the committee that was treating the TUCE, referred to as the "Pre- 
liminary College-Level Examination" by the authors • Finally, 
, they used Part I, Form A* of the TUCE feaacro section) • " r 

On the Nebraska test, the lecturele&s group achieved a ntean 
seorfc three points higher than the control group but this was not 
statistically significant. This test contained three types of 
questions — factual , concepts^!, &nd analytical • The lecture less 
grou$ achieved higher means on\all three types, but a^ain the dif- 
ference was rtot significant (*10 level) • Similar results were 
obtained on the other tests 'as we il the lectureless group doing 
better, but not significantly so. Both groups outperformed the 
national, norm group on the T'UCE, significant at the *01 level. 

The data were^lso disaggregated in terms of student ability , 
as measured by cumulative X3PA. It was expected that the poorer 
students In the lectureless group would suffer from the absence of, 
lectures and thus do less well than their counterparts £n the IfV 
group • This was not the case, however, the lectureless students 
doing as well or better than those ih the control section*. 
~* * * 

An attitude survqy showed that the lectureless group gave a 
significantly higher rating' (at the .001 level) to the instructional 
method, and had a slightly {but not significantly) more favorable 
attitude toward economics as a discipline. 

I 

Comparisons were, also made between the lectureless group* 9 
performance and that of students, exposed to live lectures in large 
and small groups during the previous year* Again theJLectureless 
<group did just as well. McConnell and iamphear concluded that 
"pedagogical capital (study guides and programmed material) can be 

successfully substituted for text-oriented lectures * (P. 31.) 

Tjhe student is forced to accept responsibility for his or her own 
learning when denied "professorial spoonfeeding," must ake the 
initiative in the pursuit of knowledge, and becomes better pre- 
pared for life-long self-education. Furthermore, the authors feel, 
tHe gap between undergraduate and graduate education may be narrowed 
if students are required to work independently. 



********** 



Mellish, G. Hartley, and Diane Bostow, The Experimental Analysis 
of Learning 3ehavior in Principles of Economics, Unpublished paper, 
Tampa: University of Southern Florida, 1975. 9pp, 
# 

This paper reports on an experimental program involving a- 
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^aodified Keller system, or personalized system of instruction. 
(See F.S. Keller, "Goodbye Teacher .\. •'" Journal of Applied 

-Bahavior Analysis , 1968, pp. 75-89.) The six programmed volumes # 
ol the text Economic Analyses Washington, D.C.s Educational* 
Technology Corp.)* were ased *at the University of South Florida 
with over 1300 students in 1974-75. OThe authors* experiment with 
remedial testing has already been reported, (See Bestow and v 
Mellish, above.) In addition, students in traditional lecture , 
classes were compared with, those using the programmed approach " . 
~ the Test of Understanding. in College Economics (TUCE) beihg . 
used f oxT pre-. and post-testing. The Karstensson-Veauer attitude 
survey was also used on a pre- and post 4 - test basis. , (See L.. 
karstensson and R. Vedder, **A Uote on Attitude as a Factor in 

'learning Economics," The Journal of Economic Education , Volume 5, 
Number 2, Spring, 1974.). There was no significant difference in i 
performance on the TUCE, but staden- attitudes towar 3 * economics 
improved in the experimental section ^tyhile those i? the traditional 
sections -declined) , and the , personalized method reduced cost per 
student, by about 50%. Furthermore, students for whom. English is a 
second language were abie to- perform "satisfactorily. " . Finally, 
there was a comparison between, criterion referenced and norm re- 
ferenced grading. Ther^was little difference between the groups' 
test scores , but the norm,-ref erenced group had far more complaints • 
about the grading system. The authors, concluded that "The knowledge 
an*? understanding of economics . gained by students in this system 
was at least as great as that achieved by students in traditional 
. lasses/ but at a much lotfer cost to the University." (P. 9,i For 
more detail, and for information on other studies at the University 
or South Florida; see G. Hartley Mellish, Diane Bostow, Nick Mystic, 
and Darre,* E. Boston, "A Behavioral Approach to. the Teaching of 
Principle^ of Economic*." Atlantic Economic Journal , November, 1974. 
An unpublished 53-page c*peT~wTXh the same title is available from 
the authors at the University of booth* Florida in Tampa. 

* 

********* 



Negus, P.E., "Individualized Learning and Economics," Economics : 
The Journal of the Economics Association , • Volume XII, Part 2, 
Summer, 1976, pp. 91-9b. 

Two economics instructors at the Arnold and Carlton College 
of Further Education in Nottingham, England, have introduced a 



♦These are often referred, to ay "the Sterling Institute Materials." 
(For other studies of these materials ,• see Craig and O'Neill, . 
above, and Soper and Thorton, below.) 



system o£* self spaced instruction ift the introductory principles 
course* The syllabus is broken down into "blocks* or units Of 
work* Each of the units contains 'basic information, references 

to textbooks , cassette tapes, newspaper articles, exercises 

multiple choice questions, essays, and topics to prepare 4 for in 
tutorial sessions. Students work through the units individually 
or in small groups, and then present their work for marking and 
discussion. 

Students work at their own pace and in whatever sequence they 
prefer, except v»here mastery of one unit is essential before 
studying a later one. 'Some form of assessment is included in mo£t 
of the units, usually consisting of multiple choice questions plus 
several problems requiring .practical application. There Is no for- 
mal classroom instruction; the instructors serve as tutors for in- 
dividuals or sjnall groups. After completing *a bloqk the student 
rtaets with the tutor to discuss the content an£ the questions. 

Evaluation was rather informal during the first year, but it 
was concluded on the basis of student re&ction and test perfor- 
mance that about half favored the new apwroacli and that "'able stu- 
dents may perform better under individualized learning conditions 
than more mediocre students." (i. 96.) The lack of whole group 
discussion guided by the instructor was seen as a weakness, but the 
tutorials enabled the instructors to identify the weak points in 
each student's understanding and provide a remedy. The students 
read more, wrote mora essays, and completed more* exercises than 
those in the conventional classes. " The instructors felt that their 
relations with students improved, but that they were compelled to 
do*iraore work because of the students* greater output. 

• ^ • . 

*********** 



Newton, Jan M. , and Fred J. Abraham, An Exploratory Study of the 
Effectiveness of Alternative Instructional Procedures in the 
Teaching of the Principles of Economics. Unpublished paper* Eugene: 
University of Oregon, 1973. 25pp. + appendices. 

'j 

Newton and Abraham compared their "interview method" with 
students taught in conventional large lecture sections and with 
those taught in small discussion groups of 15 to 30 persons. The 
large lecture section was taught by a veteran professor with an 
excellent reputation for teaching. Two mid- term examinations, a 
final examination * an M all section tinal" were administered in 
the large lecture section. The small discussion groups were taught 
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by graduate students who were permitted to select their own 
^teaching techniques, 

, Students in the interview, section did not take comprehensive 
tests chring the term, but took the "joint final *1 at the end. 
After studying from 12 to 20 pages, of the assigned textbooks, the 
students would be interviewed by undergraduate teaching assistants 
(students who had passed the course at the "V level) . The inter- 
views would last about 15 minutes, and the student's discussion of 
the text material would be j-iJdged "satisfactory" or ^"unsatisfactory** 
by the teaching assistant. This would then_be fecrfded on the in- 
dividual's progress sheet. If^an "unsatisfactory 1 judgment was„ 
made, the student would be, permitted to repeat the interviews until 
"satisfactory" #as achieved. Upon achieving a "satisfactory" rating 
the student would become an interviewer JEor other studentsr. After 
two interviews as student and two as listener, the individual would . 
take a written test on the material covered. This Was a 20-minute 
open-book examination. All questions, had, Ao be "functionally cor- 
rert" if a rating of "satisfactory" was to be achieved. Otherwise, 
the student had to re-study the material until a "satisfactory" 
rating was obtained. Alternative sets of questions were available 
for the re- testing. Individual tutoring was provided for students 
who had problenfc. The student's course grade depended upon the 
number of units mastered. Thus, the stuqent could, in effect, de- 
cide whet grade to achieve and work accordingly $ 

The iteiros i the "joint final" were selected from the TUCE. 
This, examinlatior counted , toward the course grade in the large lec- 
ture 'sections •a. 1 d the small group sections, but not in the interview 
sections. The students in the interview sections ware Required to 
take the test but- were told it would not count. . This proved to be a 
mistake, as students often left within ten minutes. This was later 
modified so that the final examination, score did have some effect. 
It could not be used to raise a student's score, but pcor performance 
on the test might lower the individual's grade. 

Revisions were made in the interview format each time it was 
employed, and the researchers did not control for possible differ- 
ence^ in age, sex, academic ability, and the like. Students were 
not randomly assigned to the three instructional groups, but the 
authors considered the students to be "reasonably matched*.!! (P. -4.) 
The fact that in -the interview section there was 1 a high proportion 
of A's and .B's led "to some disenchantment among students and faculty. 
(£. 13.) TUc Economics Department became "concerned about the' prob- 
lem' of equity caused by the huge and imposing disparity in grade 
distribution in two sections of the same coursp." (P. 13, footnote.) 
To rectify this, .the authors required students working toward an A' 
grade to submit- additional papers. - * 



f Newton and Abraham claim some b^hefits for their interview 
system*. They write: "...our extensive contact with students in 
the interview section convinces us .that they learn a good deal 
more than in other conventional teaching approaches. ... tt *(P. 13, 
footnote.) They also assert: "In a course that is not individu- 
alized, the confusion caused by a poor text may go unremedied. 



We think this explains, in. part, the relatively poor showing, mad^ 



<C — ' ~ ' <C , 

by the_ lecture section...." (P. 12.) 



, . Finally, student^ ratings of the interview section were higher 
than those of the conventional sections. For example, 37 percent 
of one group were "very delighted" as compared with 'only 15 per- 
„ cent in the lecture sections. The students also worked harder but 
. thought the course was easier in the interview sections. 

{Also see Fred J. Abraham and Jan Newton*, $he Interview 
Techn ique as a Personalized System of Instruction for Economics; 



The Oregon, Experience. Unpublished paper. Eugene: University of 
Oregon, 1973. }\ 



********** 



& v Connor, William C, A Pilot Study on the Use of Programmed In- 
struction in the Teaching of Intermediate Microeconomics. PfcuD. 
thesis* Boulder: University of Colorado 9r 1974*. Pp. 200. _ 

1 Concerned about the paucity of research in the intermediat 
courses, O'Connor devoted his doctoral study to an experiment wxch . 
programmed instruction at the University of Missouri, Columbia, 
involving about 100 students in the intermediate micro course. * 
Three, of the nine sections of the course were chosen at random, « 
> yielding usable data on 97 students.^ 

There was one control group, receiving conventional instruc- 
tion, and two treatment (experimental) groups. Treatment group num- 
ber 1 used programmed materials exclusively, while r umber 2 used 
programmed material in conjunction with the traditional lecture 
method. The treatments were randomly assigned, a*id students were 
not permitted to choose between treatments* 

O'ConiiOr chose a ^non-equivalent control group design*. {See 
* Dapald Campbell and Julian Stanley, Experimental and Quasi- 
Experimental Designs for Research . Chicago: -Hand McNally, 1970. ) 
In this design, groups are pre- and pqst- tested but *they do not 
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have jpxfe-experimental sampling- equivalence^ " .(B-_81».l^ Bart 2, 
Forxas A and B of the TOCE were used, plus a common fin^lexam 
(not developed by the researcher or any of the course Imtrtfctors) , 
and a t;est of attitudes toward the course . The pre-test mean 
scores were almost identical , and the students were homogeneous in 
terms of other characteristics.. The students were told that their 
TOCE score would count toward their grades. 

Sixteen independent and four dependent variables were* included^ 
in the .multiple regression analysis. Stepwise regression was used 
to choose independent variables which provide the best prediction 
with least number of independent .variables. The fcfUr % dependent vari- 
ables were: TUCE post^-test score (Part 2, Form 3) , simple gain scpre 
(post- test minus pr^-test) , gap closing score, and final exam score 
(40-item objective test) . The independent variables included class 
standing; major field;* whether the course was elective or required?* 
mathematics background of students high school economics background; 
pre-test score on TUCE. (dwomy* — upper or lower half on Form A) ; / 
SCAT score (dummy — upper or lower half); GPA; student' $ expected 
grade; student xnterest in economics; attitude regarding importanpf % 
of economics; opinion on .whether economics should be required; u£#r ^ 
of programmed instruction dnly; use of lecture with programmed 
struction; convention*^ lecture metTiod; and* a $ummy for split^h^Z^es 
of each treatment. - '* <* 

6 t Connor found that the GP£, pre-cQur£e interest in economics , . 
anU score on the TUCE pre-test were significant in predicting Stu- 
dent achievement in the intermediate micfo course. The pre-TUCE 
score wes by far the most significant , however. The treatments were, 
n<>t significant in explaining the dependent variables. ThT^ is, all 
three metjjp4$ were about equally effective in teaching the course, 
at least when considering the results in the aggregate. But O'Connor 
also divided the students into upper and lower halves according to 
their pre-'fUCE scores. This revealed that programmed instruction 
was most effective for the poor (|ower half) students*, while *the 
better (up^er half) students did better witb programmed learning plus 
lectures. O'Connor warns tha* his researches confined to onLy one 
college and that his study was considered to be a pilot. His final 
chapter is ^devoted to suggestions for further research with a pro- 
posed design for such, research. * ' % 
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Paden, Donald W. , and M. Eugene Moyer, "The Relative Effectiveness 
of Three Methods of Teaching Principles of Economics/ 1 The Journal 
of Economic Educatio n. Fall, 1969, Volume 1, Number 1. Pp. 33-45. 

In this study at the University of Illinois oh£_.group of_ 



4 students was taught by televised lectures, a second group had a 
live* instructor, and the third used a programmed textbook. The 
first two groups used a conventional principles textbook; the third 
used only the programmed material plus the outline-study guide which 
was given to all three groups. Lectures were given twice a, week in 
the TV and live lecture groups, supplemented by a quiz-discussioi) 
session under the direction of a graduate assistant. The programmed, 
t>ext group also had quiz sessions and were given the opportunity to 
meet with a staff member who would answer any questions they might 
have. (Few took advantage of this.) To minimize the differential 
effects of ti\e six teaching assistants, each taught a TV section 
* and a live or 'programmed section. 

An examination developed by the authors was used as a pre- test. 
, Later, the students took IForm B of the Test of Economic Understanding 
(TEU) , published by Science Research Associates . They also took a 
189-item multiple-choice test covering the concepts included in the 
course. A questionnaire on attitudes toward the course and method 
of -instruction was also -administered. 

* The programmed jgroup- achieved higher scores on the tests, both 
before and after adjustments were made for such variables as pre- 
test performance, major field, hours spent in studying econouics, 
and scores on the American College Test (ACT). The differences were 
-not statistically significant, however. 

Paden and Moyer advise other researchers to consider class, 
pre-test score, major field, credit hours completed, grade point" 

• average, number of absences, and ability when analyzing group per- 
formance in economics.^ They also measured the impact t5£ ;the six 
teacher assistants but "found no significant effect, possibly because 

. each student htad a complete outline. and detailed lesson objectives. 
\ (These teaching assistants did influence student attitudes toward 

* the course, however.) - J * 

The authors make aji important point regarding the use of the 
attitude questionnaire. As they put it, "one cannot know whether 
attitudes influenced test scores independent of the kind of instruc- 
tion received or whether the form of instruction received influence 
attitudes." (P. 38.) Thus, they presented attitude indices both 
as independent variables helping tu explain test scores and as ,de- 
pendent variables whose variation is perhapfe explair.ad by ±he kind 
of instruction received. Their conclusions- about attitudes arc -con- 
sidered tentative. Students m the programmed group had a lower 
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average index of attitude toward the course than those in the 
other groups, but the difference was not statistically signifi- 
cant. Live instruction was more popular than the other two 
methods . 

The authors conclude that the three methods were about* equally 
effective, but that TV and programmed learning "would seem to be 
much more efficient than live instruction with its 13rge expenditure- 
of teacher time* 14 (P. 45*) (Also see Paden and Moyer, "Somfc Evi- 
dence 'on the Appropriate Length of the Principles 'of Economics r 
Course," The* Jour nal of Econ omic Education * Spring, 1971, Volume 2, 
Number 2, Pd. 131-137.4 



* * * + 



Petr, Jerry L. , Experimental Use"\p£ "P.S*!. 11 or the w Keller Plan" 
in Principles of Economics. ^-£>ubXished paper, Lincoln: University 
of Nebraska, 19'* 4. 15pp. ~n 

Petr describes an expeni^ent in which two sections o£ an intro- 
ductory ***jcroeconomic* 4 course were exposed to different instructional 
methods, tctr taught both sections „ assigned 'the same textbook, and 
gave the students no prior knowledge Lnat they were involved m ai; 
experiment. Each section contained about the same number qf students 
(between 80 and 90). There j*as no statistically significant differ- 
ence between their mean gra^WjJbmt averages • The control section 
was 'taught by conventional metKodVs the experimental group was ex- 
posed to P.S.I. The experimental stupents were tested and compared 
with the controls in terms of relatives gam in economic knowledge 
and m terms of attitudes toward the subject matter and* the instruc- 
tional technique. Forms A and 3 of the TUCE (Part 1} v.ere used on 
a pre-test and post.- test b«?sis. Attitudes were measured by an at- 
titude . questionnaire and a course evaluation form. 

The course content was divided into 22 teaching units. A study 
guide was prepared for each unit, giving the objectives and assign- 
ments in the text, workbook, and book of readings tha% related to 
those objectives • Two cr more "mastery tests" were prepared for s-Ach 
unit. * Complete mastery of each unit was expected, and students could 
appear at various times to take the examinations %fi ~& h t_«»stihg room." 
If a student failed to master the material, he or she was expected to 
stpdy again and take a different form of the test later. Upon de- 
monstrating "requisite capability," the student was given the study 
guide for the nex' unit. Mid-t^rms and finals were also given. 
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^ Criteria for course grades were established, To get an V, 
IqT^b^b^^ the student had to master all 22 units and achieve 
a , score or^i percent on tRe^Hnal. In the control section , where 
the lecture-discussion technique was use&, the students took two 
"hour esams* and a final, and wrote ~paj»er. Students .in. both 
groups took the appropriate parts of the TUCE, pre and po3%, but 
their performance on this test did" not influence final grades* - 

The performance of the P. 3. I. students vas significantly 
better than that o£ the control group. Their pre-test % means were 
identical (12. ?5J~ but fehe P.S.2« section ended with a &ean of 
21*20 9 While the control group achieved a post -test- mean, of only 
17*57, Furthermore , the P.S.I, s tuaents 1 ^aiTv was^reater than 
that of the national norm group -{r.45 as compared with $.62t* 
Course grades also differed. In the P.S*X. section students 
attuned *h* level performance, while only 15 of the conventionally' 
t£ughfc v students reached this level. ' * 

In terms of attitudes, the P.S.I* students felt they had 
learned more, and. in other ways gave the course higher evaluations • 
These higher ratings were statistically significant for "General 
self-development / "Developing communication skill," "Pursuing own 
interests, " "Understanding- intellectual-cultural* values, " tt Instruc- 
tor^ interest in students," * Fairness of grading/* and "Quality of 
text 'and learning material. 5 * Petr concluded that P.S.I; "is a 
teaching technique with merit" {p. II) and that it "can be an ef- 
fective and popular technique for generating stadent mastery of the. 
content of Principles of Economics* w CP. 14} > 



** * 



Roo;*. , Joseph H. , "Contingency Management in the Teaching of Economics 
Some Results from *m intermediate Microeconomics Course/' Inte r- 
mountain Economic Review, Spring, 1973. Pp. 53-71. 

* ; ; ^ 

Roopr Joseph lA. , The Application of Contingency Management to the 
Teaching of Economics. Unpublished paper. Pullman? Washxngton State 
University , K n.d*?p.l5. - . ^ 

• F* Skinner defined teaching as the arrangement of contin- 

gencies of reinforcement under which students learn. Roop used 
the contingency management approach in an intermediate micro course 
at Washington St^te University. (Note that few economic education 
studies have Involved the intermediate level.) This approach is 
based^upon the psychology of operant conditioning which establishes 



a correlation between some aspect of behavior jtfnd reinforcement 
or punishment. Reinforcement is contingent upon certain specific 
responses, hence the term "contingency manages^*' t.^_ 

First, the instructor identifies the responses he or she wants 
to reinforce; second, a reinforcing stimulus is identified (such as 
a course grade),- and finally, specifies the contingencies which come 
between .response and reinforcement, A common practice is to specify 
a wSHBe'jLof points, to.be awarded upoii. successful completion of a 
unit of work, base the grades on numbers of points, require that a 
high leve3s.«f mastery be achieve^, and award the final course grade 
on the. basis of the total "amount of work. The total ajaonnt of t work, 
tnen, is the reinforcer — the, factor "determining the final grade,. 
Many variant forms are possible for a contingency managed course. v 
Student progress can be student-paced or dan be boffCrolied «by the 
instructor. 

. / * \ ?** ~ 

When used in other disciplines, Roop maintains jth&t this ap- 
proach resulted in superior S^udentTperformance $nd was preferred 
by the students. He attributes this to the f act *£hat requirements 
for each unit are specified, grade requirements a^e' clearly def ined, . 
?md the student receives immediate reinforcement after each unit. / 
Hoop 1 & intermediate micro course was designed to give students ex- 
perience in reading textual, material, summarizing information, 
evaluating arguments, developing bibliographies, presenting oral / 
arguments, engaging in research, and waiting reports. Students / 
could ,4o only a few of these* things and still achieve a passing 
grade* For example, studying the text and summarizing several sets 
of readings might result in a W C. M The te$ weeks devoted to these 
activities were instructor-paced, For^a 4 "B n a student would have 
to read professional literature, develop bibliographies, and /give 
an Oral report* Those wanting an "A" had to formulate hypotheses 
pn controversial issues, examine and weigh evidence, and w^ite a 
"export on their findings* These, activities were entirely /Student- 
paced an4 werJ" engaged in during the last six weeks of the course. 

The first part of the course was divided into ten /'blocks- A 
quiz was given at the and of eaoh block, and any studeht missing 
more than 2(3 percent <|i the guestions would have to Retake the test* 
Although this portion of the course was instructor-paced, students 
could make choices about the] combinations of readings they would 
use. If a student desired ^dthing more than a*"p l he or she could 
stop at the end of thi^ part o£ the course* To y evaluate the ap- 
proach, Roop gave an examination "at .the beginning of the first block 
ana at the end of the tenth, comparing ~the Results with those a- 
chieved by students covering the same material in the traditional , * 
manner. (The exam wa§ written by another instructor-) The experi- 
mental group scored significantly higher (at the -01 level) ♦ Roop 
cautiously concludes that contingency management 'is superior to 

5Cy 
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conventional methods, but notes that there were many uncontrolled 
factors in the experiment • Roop also considers Kelley l s TXPS 
(described above) to be an example* of *the application of operant 
principles to economic -education. 

This study suggests that some of the individualized or self- 
.pac,ed techniques used in the introductory course can also watl^at 
the intermediate levfci. Obviously , however, much more research . 



is needed. / 
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Siegfried., John J., and Stephen H. Strand, "An Evaluation pf the 

•Vanderbilt-rJCEE Experimental PS^ f Course in Elementary^ Economics, l \ 

The Jdurnal of Economic Education , Fall, 1976, Volume* 8, Number 1, 

pp. 9-26. a ; *' * 

One of the most interesting things about this study is i^iat 
the authors address themselves to some of the problemspnoted T5y 
other researchers. (Possibly this is because the Siegfried- 
Strand paper is one of the most recent on PSI.) They use^ a spe- 
cially designed test instead, of the TUCE, for example,* they consi- 
der costs a$ well as benefits, and they measure the impact of the 
P*S.Ii. experience on the student proctors. (Note the questions 
raised by Spector in his second .paper , .^described below. ^ Further- 
more, this 43tudy is a follow-up to the work of Fels, described 
above . 

9 

This* paper deals with the experimental P.S.I, course at 
Vanderbilt University in Nashville, Tennessee, as it was, taught 
in the fall of 1974 and spring of 1975. First the researchers re- 
port the effects on student learning. Two tests were designed for 
this purpose — a 25-item multiple-choice "Test of Elementary Eco- 
nomic Analysis" (TEEA) containing eight recognition-understanding 
questions and 17 application items, and an essay examination of 
three questions. By constructing tests tailored to the 'course they 
avoided the problem often encountgred by those using standardized 
instruments; that is, questions on material net covered J^n the 
course. On the other hand, this prevents comparisons with a national 
norm .group. - ; . ~ /' ^ t 

The TEEA was administered on a pre-test, post-tpst basis. 
(Unlike the TUCE, no alternate forms were available.) Scores were 
also obtained for, each of the three essay questions. # The TEEA. plus 
the essays constituted the final exam. In addition to scores^ on 
the TEEA (pre and post) and scores for each of the three essa^f-^ 
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questions , four summary indices' were constructed* The first of / 
these four indices — called Yg — was the final exam score, with 
the TEEA and each essay weighted equally.. The second, Y7, was 
the slim of the essay scores, with each weighted equally. The 
third, Y$, was the difference between TEEA post-test and pre-test 
scores (simple change score) . Since Yg is biased against students 
achieving, high ptre-te^tf' scores , the fourth (Yg) was a gap-closing 
score. (The gap-closing score is obtained by dividing the simple, 
change. score by the student's gap, which is the difference be- 
tween the pre-tejst score and a perfect score.) Students in both 
the P.S.I, and conventional (control) sections were tested, and 
data on student age, sex, grade point average, SAT scores, ex- 
perience in quantitative methods course, parents' occupation, and. 
number of concurrent credit hours were obtained. (Parental oc- 
cupation was a binary Variable — business-related or nonbusiness 
related occupation.) Students were permitted to choose their- 
sections, P. S.I. or conventional. (In some other studies students 
were not given this. choice. ) Students with low GPA 1 s tended to 
avc$Ld the P.S.I, course, which was reputed to have high standards. 
The results? Only iii the third essay question (requiring an 
analysis of two alternative fiscal policies in early 1975) did 
the P.S.I, students reveal statistically significant superio^icy. 
This may reflect ,the emphases upon policy analysis in the P.S.I, 
course. There was no significant difference between the experi- 
mental and control groups on Y7 (overall essay performance) , Yg 
or Y9 . Multiple linear regression analysis showed that only SAT 
stores a A _3 GPA weire statistically significant in affecting student 
learning of economics. P.S.I, "apparently . neither hfelpsnor 
hinders that degree of learning, 11 Siegfried and Strand conclude.' 
(P. 14.) A*possible explanation, howe\ r, is t that the unit tests 
were too difficult and thus reduced student self-confidence and 
motivation, according to the authors. 

The P.S.I, course, which at Vanderbilt also included the cese 
study approach, was expected to have the effect of improving per- 
formance in later courses as well. P.S.I, students were compared 
with' control students taking the second semester of the principles 
cpurse.^ That is", some of the students in the second part of the 
course had beeriP»S.I. students during the first semester, while 
others had had conventional instruction. Controlling for the 
initial level of economic understanding, the researchers found no 
significant difference between the performance of the two groups. 
"If there were any difference in subsequent learning between P.S.I 
and conventionally trained students that derives from different 
study habits or analytical skills developed in the first semester 
elementary economics course, j;t did not persist even through' the , 
very next course." (P. 17) N ^ 

Student perceptions of the course were high. An evaluation 
questionnaire revealed that the P.S.I, students gave their course 



significantly higher ratings for organization of the course, 
usefulness of assignments and reading, increasing their ability 
to, think, evaluate, and criticize, and contributing to their 
overall learning experience. The..P.S.I. professor received a 
more favorable rating than the control instructors, and the P. S.I. 
exams were seen as being more instructive and fairer- 

The P.S.I, students devoted more time to the^course, but thi-s 
was not statistically significant. The s£udent proctors, however, 
spent much more time on the course than they would have spent in 
taking an upper level course themselves — twice as much in some 
cases. (The proctors received three academic credits for their 
wofk.)^ The proctors wer s e jui^ors and sehiors with good GPA's, a^d 
all 21 of them thought they learned more from proctoring than tbey - 
would have learned in an upper-level course. The 100-item eco- 
nomics CLEP test was administered to ten of the proctors and a 
control group of 20 matched students taking , higher level courses. 
The resets substantiated the proctors' feeling that they iearned 
mote economics. (The CLEP was administered on a pre- and posi:-test 
basis, and'the gap-closing model was used.) If the cost to paroc- 
tors was high in terms of time spent on the P.S.I, ccmrse, the 
benefits were even higher. Proctoring was related to a 20.8 per- 
cent increase on the CLEP, while taking an advanced course resulted 
in &n increase of only 7.8 percent. , 

Start-up costs are high, for a P.S.I, course, but even beyond 
start-up coats the instructor at Vanderbilt felt that he was de- 
voting more time to P.S.I." than he would normally spend on a * 
three-credit course. * . 



Finally, the effect on interest in majoring in- economics was 
measured. Fourteen 'percent of the coiventionally taught students 
elected to .major in economics as compared with 12 percent of the 
P.S.I, students, but the difference was not statistically significant 

•Although Siegfried and Strand consider their results "A$>s- 
couraging," they justifx the continuation of the course bec^ase of 
favorable student evaluations of it and the benefits enjoyld by 
the upperclass proctors. /„ ** 

/ 
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Sppej,, John C, "Programmed instruction in Largg-Lecture Courses," 
The Jcjurnal of Economic Education .. Spring, 1973, Volume 4, Number 2. 
Pp. 125-12-0-.-- : : • ^ 

^oper points out Uiat the use of programmed, instruction, when 



used\to complement conventional techniques in the principles 
course, frees the instructor from the task of drilling students 

J?£3^^terial that is largely definitional and mechanical and al- 
l^^^^J^or her to concentrate oh more complex learning situa- 

^a^t^^-^llows "greater adaptability to individual differences 

rij^JX^r^l^ig^^eed-.^ ( P . 1 2 5 . ) 
- * 1* 

f|- " , In this study, Bingham's Economic Concepts: A Programmed^ 
Approach (McGraw-Hill) was used in conjunction with lectures in. 
a on^7 semester course enrolling about 100.0 students at the Univer- 
sity of Missouri (Columbia). Nine unit quizzes made up of five 
multiple-choice items were developed, with five forms fox* each 
quiz. Students "failing a qui? would continue to take different 
fqrms untiJL one was passed with 100 percent mastery* Sach "pa^s" . 
gave the .student a "bonus point" which was added to his or her 
exam and lab average in determining final grade. . 

When 'asked to rate the materials, students on the average 
ranked /the prbgrammed quiz package first. The "hybrid TUCE"* was 
used for pre- and post-testing. Wi,th the post-TUCE\,score being 
used as the dependent variable, the absolute improvemenc model 
^indicated a statistically significant relationship (.01 level) be- 
tween points earned and post-TUCE score. Indeed, about 12 percent 
of the post-TUCE score was explained by the.bonuS point variable. 
There was a negative coefficient for tne number of attempts made 
at passing tho quizzes, probably because students who had to try 
several times vtere not as bright as the others., 

• The* gap-closing model was also used; this to compensate for 
the ceiling effect of the 33-item test, tieie, ,the^ dependent vari- 
able is actual improvement (post-test score mirf3s£ pre-test. score) 
divided by potential improvement (.3 3 minus protest score) This 
indicates (in percert^age terms) the extent to Which the student 
closed the- gap' between the pre-test score and a perfect score % 
Here again the number of quizzes passed i's highly significant — . 
each bonus point closing about two percent of the gap. 

Finally, overall student performan6^ was regjressed on the 
explanatory Variables, revealing that the programmed instruction 
quiz package did have a positive impact on the varied measures of 
students performance* * 



*The 0 hybrid TUCE" is a 3 3- item version of the original i32-item 
Test of Understanding, in college Economics (TUCE) . The original 
TUCE appears in four parts /^sach of which has 33 items. The 
"hyUrld" w4s designed for research and evaluation purposes where 
one wants to cover the same basic concepts but needs a shorter and 
more manageable test instrument. .For further information see 
Phillip Saunders and Arthur L. Welsh, rt The Hybrid T0dE:^Origin, . 
Data, and Limitations,' 1 The Journa l of Economic Education, Fall 

*1975, pp. 13-19. 



Not^Of In £ pajper entitled "Programmed Instruction in Large- 
Leisure Cpua^lsY^ technical Comment, 11 William E. Becker, Jr., 
of the ^Uxi^^j^g^t^^ag ^Ixnrie'Sj&lia challenged some of Soper's sta- 
tistical,, teclfSi^tje^'. > Soper prepared two papers in response $o 
this — "Progrcuhmed In^tpictd^i in Large-Lecture Courses: A 
Correction, Reply, ana^eformulation, 11 afift Second Generation 
Research in, Economic 'Education: Problems of Specification and 
Interdependence." Soper acknowledged an error in the original 
dataset (wherein a student's score was recorded as 66 instead 
of 16) agreed with one of Becker'V criticisms (regarding incon- 
sistent application of Soper's multi-collinearity argument), and 
found "nearly all" of "Becker's comments to be appropriate. These 
are technical research issues going beyond the purpose of this 
booklet, however, and transcending the confines <yf the" study in~ 
question. As Soper put it in his second .paper, Becker's comments 
aire "not destructive of the conclusions in the original study." 
(Page 16 of "Second Generation Research ")* 
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Soper, John C, and Richard M. Thornton, "Self-Paced Economics 
Instruction: "A Large-Scale Disaggregated Evaluation," The Journal 
of Economic Education . Spring, 1976, Volume 7, Number 2. P,p. -81-31. 

'The so-called Sterling Institute materials" were used in the 
studies summarized above by Craig and O'Neill and by Mellish and 
Bdstow. This seven-volume set of economics material provides in- 
struction* through a series of modular learning^ units. Instructors 
adopting the material receive sets of multiple-choice examinations, 
and a guide on how to use the Materials and the tests -in the "self- . 
paced mode." Soper and Thornton confirm the findings of the Mellish- 
Bostow and Craig-O'Neill studies in terms of the effect on student 
attitudes, but n<?t in ternu> of the impact on cognitive ^learning. 
Further, they assert that there were "some methodological flaws" 
in those studies. (P. 82.) Both previous studies found no statis- 
tically significant differences in cognitive gains between control 
and experimental students* Soper and Thornton are bothered by 
their failure to use multiple regression analysis, however, asse~t- 
itfig that reliance upon tests of significance between groups (using "t" 
tests and/or univariate P ratios) "may obscure the effects of 
individual .difference^ in such characteristics as age, sex, or inV 
telligence on post-TUCE performance*" (p. 82.) Instead, they 
prefer procedures that will match individual , student tost scores 
with individual student characteristics ♦ 1 

*For details of the disagreement between Becker and' Soper see *'A * 
1 Debate on Research Techniques In Economic Education/ 1 The Journal 
of Economic Education , Fall, 197.6, Volume 8, Number l.~ ~ 



♦ -At\Northern Illinois University the authors conducted a 
large-£cale evaluation of the Sterling Institute self-paced . macro- 
economics course, 'There were four experimental., (self-paced) sec- 
tions and 15 control sections taught in the conventional manner. 
Graduate students taught these sections, under the supervision of 
the authors. The experimental sections were completely self spaced. 
Part 1, Form A, of L .he TUCE was used for pre-testing all students. 
There was a ^preliminary screening" of "the variables to enable them 
to determine which ones "should be included in the final estimating 
equation. Post-TUCE scores were regressed on all exogenous variables 
in the data set. Soper and Thornton found "substantial multi- 

collinearity among the explanatory ^variables " (P. 83.) For in- * 

stance, ^Jfive A(?T— scores (English, math, natural science, social 
science, and composite) were highly inter^orrelated. The same was 
true of college class and age. Only the ACT composite score was 
included in the final estimating equation, therefore, and college 
class was dropped ancf age retained because the latter was seen *as a 
proxy for "maturity," which was assumed to be an "-underlying variable" 
for which class and age are proxies. , 

The pre-TUCE score was significantly correlated with Sex, ACT, 
and Age. (Farrar-Glauber procedures were used here. See Dohaldl E. 
Farrar and Robert R. Glauber, "Multicollinearity in Regre£5ion 
Analysis: The Problem .Revisited, " Review of Economics and Statistics , 
February, 1967, pp. 92-107.) However, the. pre-TUCE score does con- 
tain information not correlated with those variables, and to omit 
such information would lead to the problem of misspecif ication. » To 
overcome misspecif ication problems, a proxy variable was constructed. 
(See the complete article for details, p. 85.) This eliminated the 
bias associated with the coefficients of Sex, ACT, Age and pre-TUCE. 

/Aside from their model and experimental design (which other 
researchers are urgefi to examine) , one of the most important aspects 
of the Soper-Thornton study is the disaggregated analysis . The TUCE 
questions are categorized *as recognition-understanding (RU) , simple- 
application (SA) , or complex-application (CA) . The "researchers 
divided the po&t-TUCE into three separate scores according to these 
categories. The self-paced students scored .665 point lower on the 
recognition-understanding items, and .38 point lower on the complex- 
application, but thesp were not statistically significant.. On the 
simple-applijation questions, however, the conventionally taught 
students scored 1*23 points higher than the self-paced students, 

and this was significant. 

r 

*Sopar and Thornton conclude that the Sterling Institute materials 
cannot be seen as superior. Indeed, students in the experimental 
gtoup scored 2.27 points lower on the post-TUCE, other things "being 
equal, than did the control group students. They also found that 
instructors can "significantly affedrt postscores." (P. 88.) Their 
study does noi rule out the u^e' of self-paced material, however. 



In their words — 

. "These results suggest that self-pa'ced materials can, be 
satisfactorily applied to the teaching of recbgnition- 
understanding type material with no, or very little, loss 
of competency"* In a combined programmed-instruction, 
lecture-discussion type course this would give the in- 
structor rfiore freedom to concentrate on application-type 
material, thus upgrading the competencies ^acquired by 
economics students. .-. . — With more lecture-discussion time 
made available by leaving recognitioi.-unde'r standing type 
material to programmed-learning techniques, a conscious 
upgrading of the lecture-discussion presentation to higher 
level application-type problems should result in improved 
■ •• student performance. " (P". 88.) * , f 

.v ~ , , . • • , • / 

They conclude t^iat "...a completely self-paced teaching format for f 
macroeconomics is inferior jto a well-directed, concept-oriented, 
graduate-student instructed, lecture-discussion taught course. 11 



* . $ * ** * a * * * 



Spector, Lee C. , Personalized System of Instruction and Its Use 
in Principles of Macroeconomics at the State University College 
at Buffalo. Unpublished paper. Buff ale : State University College, 
1974, 10pp. + appendices.* / 

Spectbr reports on an experimental use of the Keller P.S.I, 
method ia an introductory macroeconomics course at the State Univer- 
sity of New York in Buffalo. ' The experimental section was made up 
of about 65 students., These individuals received a syilabus ^which 
divided the textbook into eight units, a study guide for each unit, 
a set of learning objectives, a glossary, hints on how best to 
learn the material, a discussion of important topics, a bibliography, 
and a series of "thought questions. " y Ten stuuents from intermediate 
courses were selected to serve as proctors for the P.S.I, group. 

The students had to master a unit before proceeding to the 
next one. Upon feeling that the material had^b'een learned, tlie stu- 
dent would arrange for a 15-minute discussion with a proctor. When 
the proctor was convinced that the student understood the material, 
a ten-item multiple-choice examination was administered." if the 
student answered nine questions correctly he or she would go on to 
/the next unit; otherwise, an *dlernate form of the test would be 
taken until the score of nine or ten was achieved. The course grade 

*Also see Spector 1 s article "Tfie effectiveness of Personalized System 
of Instruction in Economics 11 in the Journal of Personalized Instruction 
Vol. 1, No. 2, September 1976, pp. 118-122. 



jwas based on the number of units mastered — eight units for an 
A, seven fox a Bj five for a„ C 7 _ajid three for a D. The proctqrs 
monitored student progress and* served" ^arar~tutors-r--- ^ 

The TUCE and student evaluations of the course were used to 
evaluate the P.S.I* method. The experimental group l s pre- test 
mean on Form A of m the TUCE was abo$t the same as th^t. of the con- 
trol group ~ 12.77 and 12.04 respectively. (The K»ean f or ,the 
national norm group. was 13.31.) ^ Oru the post-test (Form' B) the 
P.S.I, group achiev&d a meajoupf 21,,A%. as .compared jwith 18.53 for 
tho.se taught by conventional methods at Buffald, .and with 18.93 
for the national norm group. The difference was significant at 
the 99 percent^ level. The student evaluation questionnaire re- 
vealed that thfe P.S.I, group also enjoyed the experience, thought 
the method ..was effective, and. would* recommend it to other students 
The advantages of P.S.I. , accordihg to Spector, are constant feed- 
back for students and instructor, ability to work at one's own 
speed, possible reductions in student-teacher ratios (because of 
the use of proctors)r, more "give and take" provided by oral ex- 
aminations, mastery of the material, and practice. in learning to 
work by themselves. Disadvantages are that some are unable to 
, assume the ^responsibility for self-§tudy, the lack Qf intracl&ss 
discussion of current events, and the waste of time that sometimes 
occurs. (For a follow-up of Spector 's work, see the report im- 
mediately below.) 



********** 



Spector, Lee C. , Tne Long Run Effects of Personalized System of 
Instruction in Economics. Paper presented at the conference of 
the Eastern Economic Association, Bloomsburg, Pa., April, 1976. 
Buffalo: State University College, 1976. 13pp. 

This paper adds another 'dimension — .the residual impact . 
(lasting effect) of P.S.I. — to his earlier studies of the ef- 
fect of the Keller method in undergraduate economics courses. 
(See the Spector study summarized immediately above.) Spectot 
found that P.S.I, students did significantly better on the TUCE 
than students taught by conventional methods. He .considers his 
earlier findings important, but "misleading" because they pertain 
"only to tne short-run effects of the experiment." (P. 1.) 'In 
this paper, then, he deals with some of the lopger-run effects. 
The questions are: (1) Were P.S.I, students more likely to go on 
to take more economics? (2) Did P.S.I, students outperform control 
students in intermediate macroeconomics? (3) Were the experimental 
students more -likely to become economics -majors? 



♦ 

The answer to^the 'first question is tt no." Only 47 percent 
*o£ the, P.S.I. ' students took more economics, as compared with 55 
percent of the controls. The difference was not statistically 
significant at the 95 percent level. The answer tc^-the secdnd 
"~qu§srt±oir is* "yes^'l jyhe P^S.I. students achieved higher mean 
grades in the intermediate ~co wises, an$ the difference was signi- 
ficant at the 99 percent level. The answer to question (37 i£ 
w no M — a smaller percentage of P. S. i. student? became economics 
majors, but the difference was not significant at the 95 percent 

_JLeveX^_ . „ / - 

* 

In this experiment, Specter considered several variables* 
t.nat night have a bearing on student performance. Grade point 
average was used as a proxy, for intelligence. Whether or not 
students were economics majors was the proxy for motivation. 
Scores on the TUCS and grades in "the iptroductpry macro course 
wer£ used as measures of knowledge acquired in that course. , 
These variables were regres.se^ against the course grades in the 
intermediate macro course for all P.S.I, and control students. 
A diimmy variable represented th§ method of instruction (P.S.I, 
or conventional) ; and another dumny variable distinguished those 
who t had taken Money and Banking from those who had not. Ohly 
grade point average, being an economics majcir, and P.S.I, proved 
to be significant {at the 95 percent level) . GPA was most signi- 
ficant. The regression did show, however, that taking the intro- 
ductory coarse v by P. 9 r I. would improve a student's intermediate 
grade by almost one-h'alf a grade point. Thus, Spector concl-uded 
that "students receive a better foundation for Intermediate Macro- 
economics 1 * if they study the ''introductory material through the 
P.S.r.. method. (P. 7.) 

i 

Spector notes that* there are st r ili some unanswered questions. 
What are the long-run effects on the proctors? How does teaching 
a P.S.I, course affect the instructor? DO P.S.I, students retain 
their knowledge linger than students taught oy traditional methods? 
Is the student's wirk in other courses affected? 

In a discussion J^f this paper, -Alan J. Donziger of Viilanova 
University praised Spector' s work but raised some questions ais 
well. He noted tbr^t the differences between the P.S.I, students 
and the control j^roup in terms of becoming economics majors And 
in taking more Economics courses would be statistically significant 
if one accepted Vhe 85 percent level. . He did not conclude from 
this that P..§.I. is "scaring away economics majors 11 or detering 
students from> iking more economics, but he thought that a consis- 
tent pattern of similar results would bear watching. Spector' s 
assertion that P.S.I, provides Better preparation for work at the 
intermediate macro level was seen by Donziger as "reasonable. 11 He 
expressed concern, however, about the problem of multicolinearity . 
(See. the Soper-Thornton study cited above for further consideration 
of this problem.) A basic question raised by Donziger is: "If 
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students -fcho hatl P.S.I, principles. Courses do perform- better in 
Intermediate Macroeconomics, doe? this mean that P.S.I, helps , 
students 'earn more (somehow defined*) in future courses, or does 
it make them more proficient at taking examinations?" (Qu'pted 
from Donziges's 'paper "Discussion on The Long Run Effects of 
Personalized System of Instruction in Economics." yillanova. Pa. 
Villanova University, 1976. 6pp) . \, 



Tietenberg, T.H- »*, A Comparative Analysis of the Personalized 
System of Instruction with the Lecture JOthod in Terms of Fiver 
Altemitive Output Measures". Unpublished paper. Williams town, 
Mass.: Williams College, n.d. Pp. 22. 

Tietenberg, T-H., "Teaching Microeconomics Using the Personalized 
System of Instruction: An Evaluation," in James Johnston Ed., 
Behavior Research and Technology in Higher . qu estion < Spring fie Id: 
Charles C. Thomas, .1974). 

Tiecenberg is one of the few to report studi.es at the inter- 
mediate level — in* this case an experiment? with* P.S.I..- in" an 
intermediate micr« course at Williams College. The focus was on 
•five different outputs: (1) performance on the final exam; (2) 
performance in courses taken concurrently?- (3) performance in the 
macro theory course; (4) performance' in_sjJLfcsequent micro course 
electives; and (5) decisions on majoring in econdmic* and political 
economy. The lecture format was compared with P.S.J* 

Tne P.S.I, students could proceed at their own pace (within 
limits) and had greater flexibility in allocating the' time spent 
on the course. The professor was available for individual consul- 
tation, which made it possible to "personalize" instruction by 
relating economics to problems of interest to the student. The 
course was divided into units, each unit .containing an introduc- 
tion relating the material to previous units and/to the course in 
general, a' set of objectives, a set of procedures, and a set of 
sample problems or questions. Upon achieving a perfect or near- 
perfect score on a unit test,' the student would go on to the next 
unit. Failing a unit test would have no effect on the student s 
graie, but it would slow him or her down. At the beginning of the 
semester the student would know what had to be done to efarn an A, 
a B, a C, and so on. • 

The 92 students participating were divided into three ac- 
tions, two of which were taught in the conventional manner. A 
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standard textbook was used, but the P.S.I, group also received • 
handout^ to supplement the text. Thes^ were designed to pro- 
vide the kind of elaboration ordinarily provided by lectures. 
Unfortunately, it was not possible to have one professor teach 
both experimental and dontrol sections^ thus differences^ in 
teaching ability 'had to be ta.Ken into account. Furthermore, 
stud 2ntg%/ere permitted to select their -preferred' section* 

*A comm6n final exam was prepared, made up of objective and 
essay questions. This was constructed by the three faculty mem- 
bers 'teaching the course. Each faculty member graded one part of 
the t^st^ithout knowing the name of the student whose paper was 
being evaluated. The final exam score was the dependent variable 
in a multiple regression analysis. This was regressed against 
several control variables. P.S.I, was a binary variable baking ' 
the value 1.0 for a student in the experimental section and 0.0 
for the control student. There was also an interaction varic*b^e,„ 
PSIGlPA*, which took the value of the student's economics GPA upon 
entry int^ the course if the student was in the P.S.I, section 
(z$ro otherwise) . A positive sign on PSIGPA would indicate, that 
the benefits of p.S.I. were greater for students witfr strong re- 
cords of past performance, after controlling for the other vari- 
ables in the equation. SAT math and SAT verbal scores were taken 
into account, as were age, sex, takiug the final exam e^rly, and 
the teaching experience (in years) of the instructor. 

Tietenberg concluded that there was "a breakeven economics 
grade point average." Ey this he meant that a breakeven economics 
GPA exists when all students above that average are estimated to 
gain. t mbre from P.S.I. , ,and all students below that average are 
estimated to gain rfiore from the traditional method. The breakeven 
economics GPA was estimated to be a C minus, in this experiment^* 
The P.S.I, format, it was estimated, led to higher final, exam 
scores than did the lecture method for comparable students, but 
the difference was not statistically significant. 

In courses taken concurrently (including those other than 
economics) , grade point average for those courses was the de- 
pendent Variable. The breakeven economics GPA in this instance 
was' slightly below a B. That is, of all the students who entered 
the course with a B average in eccncmics (or better) , those who 
were in the P.S.I, group were estimated to do better in their, 
other courses taken during the„ same semester than those with com- 
parable characteristics" who attended lecture sections. The re- 
sults were not statistically significant, however. 

Students involved in the experiment who took macro theory in 
the following semester were further analyzed, with the grade in 
the macro course regressed against the same variables used ih the 
earlier model. The breakeven point was between B and B plus for 
this group. P.S.I, students entering the earlier micro course 
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with a B plus economics GPA were estimated to have performed * 
better iri macro theory than similar students who studied micro 
under the, lecture 'method* Again, the differences were Apt ?ta-<* 
tistically significant. * *. * * ^ w *• ^ ' 

Performance in subsequent micro courses was measures by 
GPA in those. courses . Etere, the breakeven point -was l^Cween & 
minus and pl.u5 t and- although .those coming from the P.S.fl ex- 
pei;ienc^ did better than their peers if they were above the 
breakeven" point, and those beiow.it did better if they' ha<J been 
% in the" lecture sections, the differences were not signif icdn€^ r 

Finally, a tworgroup discriminant analysis was used to see 
if P.S.I, affected decisions to major in economics. This analysis 
uses a vector of independent variables to classify the sample into 
twq groups where one knows the appropriate classification before- . 
hand — in this case- the groups befr^ those who chose to majpr in 
economics or 'political economy and those choosing some other field.' 
In his unpublished paper (p. 18) Tietenberg explains: 

1 The ex post classification takes place by estimating a 
linear discriminant function and using this function to 
generate an estimated discriminant inde^. When the value*, 
of the index is higher than some critical value, the ob- 
servation is assigned to th£ first group. All other 
observations are assigned to the second group. By exam- 
ining the discriminant function coefficients and performing 
the appropriate hypothesis tests on them it is possible to 
study the effect of the teaching format on the decision to 
-major in these fields and the statistical significance of 
these results." (For the appropriate testy see George w. 
Ladd, "Linear Probability Functions and Discriminant 
Functions," Econometrica , October 1966, pp. 873-885.) 

In this .instance the breakeven grade point average v/as C. That 
is, students enterifrg the experimental course with a C averaga 
(or "better) were slightly mcqre inclined to major in, economics or 
political economy if thgy wer^riir the P.'S.I. section. Those vith 
a C minus or below were slightly more apt to major in that field 
if they were taught by the lecture method. The differences were 
far too small to be statistically significant^ however. 

Tietenberg concluded that "the P.S.I, format was neither in-; 
herently superior nor inherently inferior to the lecture format 
in any of these dimensions » " (P. 19, unpublished paper.) However, 
he was struck by the "remarkable consistency in the distribution* 3 1 
of these differences among students with differing. . .competence in 
economics prior to their taking the experimental course." (Pp. 19-20 
unpublished paper.) The students did seem to gain more from P.S*I. 
in all five of the educational outputs if they had good records in 
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their previous economics courses. .Tietenberg* thought th^rt P.^,.1. , 
therefore, might be "biased toward the 'more capable student/ 1 but 
~ S\jsges^ed further research on this point. 

1 *********** . 



Tolles, N. Arnold, and Peter J. Gimndn, "Using an Audiovisual- 
Tutprial* Process in Macroeconomics," The Journal of Economic . 
^du£ation : Fall, 1974, Volume 6, Number 1. Pp. 57-58. , ( 

^ ffhTs^udio-visual tutorial (AVT) process developed at the 
Sfcatfe University of New York College in Geneseo perjnits the s±:u- 
iH*/^ ^ent to Sear part of the responsibility for teaching, and can be 

^'^used for individual study and self-paced instruction. Usually \ 
;* the student takes a pre-test, reviews the unit Jmaterial in a t . * < 

workbook, listens 'to an audio- tape, and views ^55-mm slides which 
accompany the 'tape. Difficult sections may be repeated, 'of course, 
and thd student^ takes both a "self-instructional test" and a test 
for credit/ .(See David A. Martin, Introductory Economic Theory . 
: ^ew fc York* Mc^raw-Hifll,. 1971.) 

When* the ^materials were used at Gene^seo, it was found that the 
students could complete a standard basic macrceconomics course in 
ly eiph^t weeks. The authors report that student evaluations of 
VT were "highly positive." During a sprincj semester the researchers 
relief entirely upon s^lf-study with the' AVT materials. In the fall "* * 
thay tried a different approach, having the students meet their in- 
structor £n discussion sessions after having used the* AVT materials. 
Th£re was also "class-assisted use" of thV^AVT material*. Student 
improvement ^was greater in the fai_, suggesting that the combination 
of AVT and conventional discussion methods- may be superior to AVT 
alone k or to the traditional approach alone. The combination method 
"* seemed to be better for weak students. The authors pointed to the 
.need for more controlled experimentation, however. (See the Becker 
*and Salemi study, above, for an evaluation of the Gen£|Leo AVT ma-^. jgL-**^ 
€erial. 9 Also sefe Walstad's study, below.) f ~ ~ 
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Van Metre, Dale, Principles of Learning and Introductory Economics: 
A Discussion and an Experiment. Unpublished paper. Ogden, Utah: 
. Weber State College, 1974. 21pp. 

* Before describing ' his* experiment with P.S.I. , Van Metro makes 
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a number of interesting observations about teaching techniques, 
and research. He notes that many .studies do not a't'tempt 
jri&tch .-the modes of instruction with the particular kinds of ' 
^.earning to which they mdy<E>est be suiteck 11 He dis- 

relishes instructional objectives .and cites) research showing that 
* "student achievement in an economics ^ourjse is significantly im-« 
prqyed when instructional' objectives aroused," (P. 3,) Van 
Metre asserts that "people generally learn more when they are 
actively participating in. the learning* process , 11 (P. 6) and that 
a change in "the method of instruction has no iogical potential 
for changing student performance unless it changes what the stu- 
dent is doing." (P. 7.) He feels that "learning takes place 
within the individual according to each individual's 'internal 
conditions of learning 1 such as the desire to learn, willingness 
to follow directions, amount of prior learning, etc." (P. 8.) 
Finally, he notes that the lecture method "is likely to frustrate 
those students who would like to learn faster and will penalize 
""those students who... can't learn as "fast as the course plan dic- 
tates." (P. 8.) (Also see h'is article "A Learning Theory of 
Economics instructional Development," The Journal of Economic 
Education / Spring 1^76/ Vp- 95-1*03.) * ' : , . 

Van Metre's experiment involved the evaluation o^S^course 
patterned after the Keller method of P.S.I. Six units were pre- 
pared, each_ containing a list of objectives, a programmed study 
guide which^accompanies a basic textbook, practice exercises, 
readings on current economic issues and audio-visual media. Exam- 
inations were pjufepared for each unit. ' Students* set their own 
learning and testing schedules, but had to complete all six units 
by the end of the term. " Students could re-take the tests (using 
different forms), which might raise or lower their grades. . Note 
that Van Metre's use of the Keller method thus differs from that 
of-^ome others wherein .re-tes'tj ng^ could raise grades but not lower 
them*. He would multiply scores on the first re-take by .0.92 and 
on- the second by 0.84 ,"to encourage 'the students - to study T»ore 
between retakes and to not use the tests only as learning devices." 
(P. ^10.) Furthermore, each test .contained at least one essay ques- 
tion. Tutoring was available ddily, and unit workshops were held 
•^Cdiscuss 'current issues and the wore difficult course objectives. 
There were also "extra credit discussions" dealing with the as- 
signecT-readings on controversial topics. Each student had to -om- 
nlete all six units ?^ a specified minimum performance l^vel, wh#re 
as in some colleger students were not required to finish all 

of them. 

The experimental course was taught for three quarters rib ing 
one academic /ear at Weber State. Sections taught by experienced 
processors using conventional approaches w^re used as controls. 
Van Metre controlled for student abilities by including ACT scores 



and^ cumulative grade p6int averages, TheT equation- (using m least 
st^u^fes regression- techniques) also included a dummy 'variable 
Jror.jsetf, «imd. a dummy variable for teaching method. Part One of. , 
^^Ke-TUCE was used, Form A for pre-testing artd Form B for post* . 
_ 5 ^he" gap-closing model was employed* .This measures., the.. extent to. 
which the student closes, the gap between^ the pre-test scor$*and, r 
a perfect score, .The pre- tesir score is deducted from tfie^post-' 
test score to provide the, numerator* The pre- test score is de- 
ducted from 33 (a perfect score) to yield the denominator (This 
-is the "gap" or the student's potential improvement.) The result 
is an indication, in percentage, terms, of how w^ll the student 
closed the gap. ' (This gap-closing model is now widely used in 
economic education research, but it .is still under study* For 
example, see Paul H. Kipps, Howard k. WilheOm, ^nd Daniel R, Hall, 
"h Note on the Use of Multiple Regression Malysis in Studies of 
•Achievement in Economics," Tfre Journal of Economic. Education » 
Sprang 1976, pp. 130-132*) * * ~ • 

The regression analysis revealed that ACT score and GFA were . 
highly significant (above 99 percent) , and that the method of 
teaching was significant only if one accepts a level of 89.6 per- 
.cent. ' Students ir the. P.S.I, course would be expected to' achieve ' 
6.7 percent more of their potential improvement than those in * 
conventional courses. 

, \ 

Van Metre hesitates to draw firm conclusions from his study. 
He points to sources of Jttodel misspeci£ication r such as the lack, 
of variables to ac,cdunt for differences in teacher abilities ami 
objectives.- Ev^n if textbooks and obi actives are the same for 
all students* he note®, re&ehers .mi^tht /emphasize some objectives 
more than qthers/ He ^nks^fcBa^ Ttfig motivations, aiylt^txpg^a^ops 
of the teacher' mi\ .it influence student performance* TJP^-l*** r '-i*.~T 
Student age* marital status,, employment situation* and attitudes 
might also nave an effect. "The maturity and motivation oi stu- 
dents is important too* because more ^responsibility i*s placed upon 
students xn a P.S.I, course than in a Iv^ture course. " (IK 15*} 
Other important- points made by Van Metre arenas follows? 

—> * . *To &ay< that a given teacher will have success with* $#y, 
group discussion because a afudy showed it* successful for a dif~* 
"ferent instructor doesn* t make sens*.* *A teacher * s ability to 
instruct varies across methods.**.* {?, X5«) w Most previous 
studies have implicitly assumed that a particular method is equally 
effective for all types of learning or that there is cw.Jy one type 
of learning outcome xn **l#mehtary economics* Future. studies must 
distinguish among types of learning and do so under conditions 
£or more .teacher variables than did thxs present study." (P* 15*) 
**Fmal test performance is *v>i "the only method by 'w&ich teaching 
efforts can or 'si.au 14 b# evaluated.* rt Our ability to measure 
how well the nrlcjfcenta**/ oconomjes -oursu toacho» students atti- 
tudes tavaVd dconoaics. intellectual problem solving, alternative 
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value systems, etc* is 'very weak, 
roe.asure £Ke*" extent ,to~ which our c 
cally handle new situations using 



We also don't adequat;©|$; 
uyses teach students jio^ogl- 
unfajniTiar approaches. 11 {P. 17.) 
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tfalstad, William B., An Alternative to the Conventional The 
Audio-Visual-Tutorial Method for Teaching Introductory College 
Economics. Unpublished paper. Minneapolis: University of Minne- 
sota Center for Economic Sducation, 1975* Pp. 15. 

W&lstad's study differs from most, ethers in two respects. 
First, it is one of the few to detail. with two-year colleges. 
Second, several schools weru included in the experiments whereas 
$iost studies involve students from oply one institution. ThQ. 
study was designed to measure the impact of David Martin's audio- 
visual-tutorial (AVT) package, Introductory Economic Theory (Npw 
totki MctGraw-iBill,- 1971). (For other*' studies involving /this AVT 
package see decker and Salami, # and Tolles and Ginman, above.) 
AVT waa compared With thu conventional lecture-discussion agproach 
both in .ter&is of student learning and effects on student attitude 
toward economics. 

Three two-year colleges in Minnesota and three in Missouri 
were included in the study # a,nd usable data weie collected on 330 
students. Students at each .college were randomly assigned to con- 
trol and experimental q roups* There were no signif leant differ** 
ences between g^oijps in tgjgps.&f ageriJistribution, . se#, class, \ 
and previous col'legS worfc^. """Each instructor received i^t^m4tion 
about the study and attended a one-day workshop to learn l&' use 
the AVT package • One instructor at *;ach college taught both the 
experimental and control classes. * „ 

Tho AVT package includes 35 mm color slides, audio- t^pes^ 
and a vrorktext that includes instructional s<?lf«*study a'ids* Basxc 
mxcro sind macro theory -are covered. - After an introductory class 
session in ^hxen students were shown how to use the materials and 
operate the equipment* the students were to use the AVT package 
for the next nine sessions in place of the usual class lectures. 
PVe-tpats .and post-tests were administered with each unit, If 
perfor^a&oe on* the post-test was poor, the atudent was ins.^uoted 
to review the unit and retake the test. Meanwhile, students in 
the control classes were covering the same concepts via lecture^ 
discussion, Instructors were available to help ^ AVT students 
having problems. ' 
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Forms A and B of the TUCE were used for tasting economic 
imder^tartdDLng. NO s*|^T£loan^ differences were fbund either 
dii thcrpre- or post-tests IsetafeQtt lexpei^etital AiiS "control 
.groups. A multiple linear regression analysis was pade, with 

age*, pre- test score , post-high school work in economic , 
Whether pr not economics was, required? ^ |, re-course interest, in 
economics, college attended , and being in a control or experi- 
mental group as the regressoirs* The post-test score was the 
d#|^dej!ft variable* T&e pre-TUCE variable was a significant 
contributor i<y*sfcn*«ant learning, and Significant differences \. 
%ere found between a^l the colleges in regard to student llsarn- 
itig off economics ♦ Wai^tad was unable to explaih the college J 
difference** .Thus, Warstad does not support the Tolles and ' * 
Gjb^n ^findis^f fchatr: the AVT progri^ was, more effective than con- 
ventional insi^ Giving it a more positive interpretation , 
however-/ he points out that his findings do indicate that stu- 
dents can learh economics on ^their own by j^in^f the AVS .package.* 
arid that* they *will leam m much as conventionally taught stu- 
dents « ^here was no important difference* 1 between coj^irol and 
experimental groups in term of student post-course ratings 6f 
economics. W&Istad's conclusion is that "The AV5 method inay % 
o£fer epbfcomic instructors' and students an - alternative to, the,, ^ 
conventiohal without concern over a drop in student achievement 
or attitudes towards econSmics,* (P. IE.} (For additional de- 
tails* see Walst&d**? The Effectiveness of an Audio- tutorial 
Approach* for Teaching Introductory Bcono&ics in Selected Two~Ye$r 
ColJeges* Oiipublished M.A. paper, University Of Minnesota,,, 1975*) 
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Wetik^ in th 3 Economic Psin- 

yTcipl&s Cpncse* Richmond,. .Virginia; Virgfinia Commonwealth Univer- 
V *^'£5%^ JLS?^ 14pp. ^Unpublished paper, submitted to The Journal of 
%*> \ Economic Education,) * * 
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Wetzel's study does ,n#t concentrate on personalized, indi- 
vidualized, or self-pabed instruction,, but his conclusion does 
raise an important question about the advantages of. such instrue- 
txonJ x , 

Illustrating his point with an indifference curve, Wetzel 
asserts that an improved teaching technique (represented by an 
outward shift of the transformation curve, where "Achievement 
, in Economics" appears on the horizontal axis and "Leisure Time" 
appears on the vertical) enables the student to ieam more eco- 
nomics, enjoy more leisure (a proxy for anything other than 
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Studying economics-* , or. a combination of both. He found that 
students expected grade and final grade were positively corre- 
lated with, student effort, significant at the .05 level, and 
that day students exerted mose effort than evening students. •* * 
{FOjt more information on the "effort variable" see the learning 
theory of Richard B. McKensie^and Robert J. Staff in * their book 
m Scondmic Theory of learning , .Biacksbuig, Virginia: University 
Press, 1974 . ) He supports 8 the belief .that final grades are a 
result of both effort and ability." (P. 9.) There are implica- 
tions of his study for, xtsearch -on ^comparative teaching techniques 
fee Beels. In his . oroide, i "The incorporation ,,o£ decision making 

* theory into" the evaluation process and its implications for the 
student's trade-off between achievement and leisure ase guite im- 
portant. The real* advantage of T3EP$ f personalized instruction or 

reaching by television, may not be h&£her achievement i.n the 
course undsr consideration. The real advantage <may.be that lefi^ 
real, effort needs to* be spent on that particulars coarse which" " ' " 
leaves more student leisure time available for other* purposes the > 
student has in mind." -{P. 10.) - ' ¥ . 
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